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Executive Summary 


Under the direction of the Assistant Secretary of the Army for Acquisition, 
Logistics, and Technology (ASA[ALT]), the US Army Research Laboratory (ARL) 
hosted a series of meetings in fall 2016 to develop a strategic vision for Army 
Science. Meeting topics were vetted through the ARL Director and approved by the 
ASA(ALT). Their selection was based on their potential to dramatically impact 
military capabilities in the long term. This report is a summary of those meetings 
and their outcomes. 

The following 6 topic areas were selected: Human-Agent Teaming, Neural-Inspired 
Artificial Intelligence, Distributed Information Processing and Data Analytics, 
Materials for Sustainable and Mission Flexible Intelligent Systems, Bioenabled 
Materials Synthesis and Assembly, and Towards a Science of Complexity. 
Questions considered at these meetings included the following: 

• Within this technical area, what capability can it deliver to the military 25 
years from now? 

• What technical hurdles exist that limit our ability to realize this capability? 

• What research does the Army need to support now to overcome these 
hurdles and enable the desired capability? 

Meeting organizers were all senior members of ARL’s technical staff and include 
ARL Branch and Division Chiefs, elected Fellows, and Army STs. For each 
meeting, ARL invited a small number of world-class experts as speakers, with a 
long-term, broad view of the specific area and an awareness of its trends. The 
meetings were structured to obtain a variety of viewpoints, not just near-term DOD- 
related expertise. Target attendance per meeting was roughly 25. Attendees 
spanned a large variety of Research and Development (R&D) organizations, both 
civilian and defense. 

Based on the output of the meetings and subsequent discussions among meeting 
organizers, ARL developed the following specific recommendations. 

ASA(ALT) should invest in the following programs: 

• Support a collaborative effort across industry, academia, and the Army 
research community focused on modeling emergent team behaviors as a 
function of individual human and nonhuman members, and approaches to 
augment individual behaviors and functions to improve team performance. 
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• Support a multidisciplinary collaborative effort between the Army research 
community and academia that addresses combining neuroscience with 
artificial intelligence. Research foci include distributed artificial 
intelligence for tactical applications and coupling artificial intelligence to 
action. 

• Support a multifaceted program (internal, external, and collaborative) in 
Adversarial and Collaborative Reasoning and Machine Learning to develop 
robust, friendly artificial intelligence, and penetrate, characterize, and 
exploit adversarial artificial intelligence. The research program must 
engage both social and behavioral scientists, as well as information and 
computational scientists. 

• Support a collaborative effort between industry, academia, the Army 
training and doctrine community, and the Army development community 
to develop portable artificial intelligence capabilities (“AI in My Pocket”) 
that enable the near-instantaneous delivery of tailored context-specific 
support to Soldiers in complex environments and dynamic situations 
ranging from situational awareness and dynamic mission planning to 
embedded training and readiness. 

• Support a collaborative effort across industry, academia, the Army research 
community, and the Army analysis community to establish an analytics 
infrastructure that addresses standards, methods, and tools for training, 
testing, validation, and verification of doctrine—for example, methods and 
approaches to interpret and quantify Soldier and joint Soldier-intelligent 
systems performance for physical tasks and decision making. 

• Establish an academic center to develop the science of materials integration 
to enable the ingestion, circulation, and metabolic transpiration of energy 
and power for local actuation and computation in a distributed and resilient 
manner. 

• Support internal Army research efforts that focus on materials and subscale 
functional elements to promote rapid assembly and synthesis of materials 
across multilength scales, modeling and simulation, and fabrication 
technology to produce complex multimaterial, multifield metamaterials. 

• Establish a collaborative effort between the Army research community and 
academia to develop fabrication and manufacturing processes and 
procedures to produce multifunctional building blocks unique to intelligent 
systems. 
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• Support an extramural program in Complexity and Emergence to develop 
mathematical models for predicting nonequilibrium behavior of complex 
multiscale systems that advance our understanding beyond statistical 
physics and nonlinear dynamics. 

• Increase the hiring of mathematicians to enhance the capacity for 
mathematical analysis internal to Army laboratories. 

• Support a multifaceted program (internal, external, and collaborative) in 
Living Materials to develop the principles enabling bioassembly and 
manufacturing of complex synthetic and biocomposite materials and to 
identify factors capable of accelerating the process to reduce maturation 
time. 

• Support internal Army research efforts to enhance Army investments in 
multiscale modeling to accelerate the rate of discovery and understanding 
in biology by fusing multiscale modeling with bioinformatics. 
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1. Introduction 


In 2013, the Assistant Secretary of the Army for Acquisition, Logistics, and 
Technology (ASA[ALT]) charged the US Army Research Laboratory (ARL) to 
develop a strategic research plan for the US Army for the next 20 to 30 years. In 
the years since that initial request, ARL has engaged in an annual examination of 
capabilities that may provide the Army with an advantage in future conflicts. 
Integral to this examination is an assessment of the hurdles that hinder us from 
achieving this capability and of the latest scientific opportunities that increase the 
likelihood of overcoming these hurdles. 

This assessment, refined each year, has helped to shape Army investments in 
science and technology. For example, in 2014, the report recommended increased 
in-house efforts in quantum information and sensing in conjunction with the 
establishment of an off-base lab or a joint research institute with a university. 
Consequently, ARL’s investment in Quantum Information Science is now close to 
$15M and includes collaborative efforts between ARL, the US Naval Research 
Laboratory (NRL), the National Institute of Standards and Technology (NIST), and 
the University of Maryland. In 2015, the notion that, in urban terrain, the Army 
should “plug into the city” evolved into concepts for the Internet of Battlefield 
Things. In addition, “just add water protection” evolved into ARL’s program on 
Expeditionary On-Demand Manufacturing. 

This report provides the assessment and recommendations to ASA(ALT) from the 
meetings held in fall 2016. 

2. Technical Perspective 

ARL recognizes that armed conflict remains a contest of wills and that actors in 
conflict fundamentally seek to persuade others to accept their perspective. ARL 
also recognizes that, historically, investments in science and technology focused on 
combat in the physical domain. However, as stated in the report of the first Army 
Science Planning and Strategy meetings in 2014, the means for this persuasion, 
now and into the near future, exist in 3 realms (Fig. I) 1 : 

• physical, the domain of activities defined in space and time by the laws of 
physics; 


*US Army Research Laboratory (US). A Report on Army Science Planning and Strategy [internal], 
Aberdeen Proving Ground (MD): US Army Research Laboratory (US); 2014 Mar. 
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• informational, the domain of activities defined by thought and perception; 
and 

• cultural (or human), the domain of activities defined by the interaction of 
people and societies. 



Fig. 1 Three realms of future conflict 

ARL has focused its strategic intentions on these 3 realms and in recent years in 
overlapping areas, such as mixed teams of humans and intelligent systems, and 
operations in cyber space whose effects are realized in physical space. 

To focus further its strategic research intent, in 2016 ARL identified several 
essential research areas (ERAs) critical to enabling the future Army force. ARL’s 
ERAs are clustered in 4 groups: Intelligent Teams, which recognizes the 
importance of intelligent systems to future capabilities; Vulnerabilities and 
Constraints, which acknowledges the impact of operating intelligent systems in a 
contested and complex environment; Implementation, which addresses the 
integration of individual technologies to realize and apply autonomous capabilities; 
and Discovery, in which scientific opportunities drive the development of new 
capabilities. 

Intelligent Teams: On the future battlefield, humans will interact intimately with 
artificially intelligent systems. Intelligent Teams considers the abstract boundary 
between humans and intelligent systems across which information is exchanged, 
increasing the intelligence of engineered systems and preparing Warfighters to 
engage with and exploit intelligent systems to create an effective fighting force. 
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Vulnerabilities and Constraints: Although new information technologies can 
help commanders make more informed decisions, they also present the commander 
with new vulnerabilities, both physical and virtual, as well as limitations. 

Implementation: To ensure that the foundational advances pursued in ARL’s 
Essential Research Areas have a path to practical application, the third cluster 
considers issues related to implementation. 

Discovery: Discovery is the process of identifying, creating, developing, and 
exploiting innovative, yet Army-relevant, science and engineering advances. 
Discovery is essential to the Army’s mission. It ensures the Army’s continuing and 
future technological superiority, and creates technological surprise for our 
adversaries while avoiding technological surprise for ourselves. 

All of the fall 2016 meetings address knowledge gaps evident in the ERAs. Three 
meetings address the Intelligent Teams cluster: Human-Agent Teaming, Neural- 
Inspired Artificial Intelligence, and Distributed Information Processing and Data 
Analytics. Under Implementation, Materials for Sustainable and Mission Flexible 
Intelligent Systems considered the limitations of current materials to address the 
needs of mobile intelligent systems. 

Finally, ARL recognized that all advances in science and technology are not 
grounded in the physical sciences. The potential exists for biology and mathematics 
to enable future Army capabilities as well. However, these have not yet been 
demonstrated, which puts them in the realm of Discovery. The meeting Bioenabled 
Materials Synthesis and Assembly considered how biology can be exploited to 
improve the properties of nonbiological materials. The meeting Towards a Science 
of Complexity identified deficiencies in present analysis tools to model and predict 
the behavior of complex systems, both physical and virtual. 

During fall 2016, ARL conducted six, 2-day meetings in these topic areas. Each 
meeting addressed the following questions: 

• What capability can this technology deliver to the military 25 years from 
now? 

• What technical hurdles exist that limit our ability to realize this capability? 

• What research does the Army need to support now to overcome these 
hurdles and enable the desired capability? 

The meetings were structured to obtain a variety of viewpoints, not just near-term 
Department of Defense (DOD-related expertise. Attendance was invitation-only, 
and the number of attendees per meeting was roughly 25 but varied from meeting 
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to meeting. ARL invited a small number of world-class experts as speakers from 
government, academia, and industry, who have a long-term, broad view of the 
specific area and its trends. 

The next section summarizes the discussions that occurred within each meeting. 
Recommendations for research in each area are indicated. The full meeting 
summaries are found in the Appendix. 

3. Discussion 


3.1 Human-Agent Teaming 

The future Army mission is expected to involve high-tempo operations and 
engagements with near-peer adversaries across distributed, multidomain spaces 
that are unlike any previously encountered. Unique operational challenges that span 
a complex mix of physical and cyber domains, ranging from sparse rural to dense 
mega-city environments, will require adaptive vehicle-based Soldier-systems to 
possess a range of capabilities extending well beyond those expected for traditional 
military vehicles. Future concepts for these Soldier-systems will be influenced by 
future mission challenges (e.g., multicultural environments, significant increase in 
intelligent systems, dense urban areas), constraints and desires for specific 
approaches (e.g., distributed teams, reduced crew sizes, closed-hatch operations, 
effective Soldier-autonomy teaming), and the direction of technological advances 
(e.g., machine-augmented and intelligent agent-enabled task execution, adaptive 
and individualized Soldier-system design). 

A 2015 Army Science Planning and Strategy Meeting (ASPSM) on Individualizing 
Technology for Effective Teaming focused on identifying research areas important 
to Soldier-autonomy teaming within this mission space. The group then used that 
perspective to derive concepts of future human-autonomy teams and highly critical 
future research questions. This 2016 ASPSM on Vehicle-based Soldier-Autonomy 
Teaming augments those results by attempting to use a different perspective to also 
identify future concepts of teaming and critical future research questions. 
Specifically, this meeting focused on future Soldier-autonomy team operations, 
interactions with the battlespace, and capabilities enabled through manned vehicle- 
based Soldier systems. The group analyzed those capabilities to derive concepts of 
future Soldier-autonomy teams and fundamental research required to enable their 
realization. 


Approved for public release; distribution is unlimited. 

4 





Capabilities 


Future Soldier-autonomy teaming capabilities were discussed in 3 general mission 
contexts that were chosen to encompass a variety of operational challenges. These 
included a presence patrol in a low-density urban environment, leader-interaction 
in a rural environment, and extraction from a megacity environment. Across the 
discussions, a strong theme emerged that interface technologies, comprising sets of 
intelligent agents, hereafter referred to as intelligent crewstations, will be a critical 
technology to “team” with Soldiers while executing all phases of operations. 
Advancing technologies for intelligent crewstations will enable mission planning, 
execution, dynamic adjustment, and after-action review for improved training. 
Moreover, this foundational intelligent crewstation-Soldier team will provide an 
essential functional unit to enable manned vehicles to team with other manned and 
unmanned assets to maintain survivability and lethality during mobile operations 
across a range of environments. In addition to the intelligent crewstation theme, 
several other strong themes of future capabilities also emerged from the meeting: 

• Intelligent Soldier aides for operational training, mission planning, and 
rehearsal 

• Effective real-time maintenance and communication of situational 
awareness across Soldier-autonomy teams resilient to degraded, 
compromised, and rapidly changing informational conditions 

• Situation-adaptive reasoning, dynamic resource allocation, and task 
management to support collaborative Solider-autonomy decision making 
and performance 

• Context and state-aware autonomy to detect or anticipate operational needs 
and take necessary actions to support the Soldier-squad mission 

• Heterogeneous unit teaming, including direct interactions across mounted, 
dismounted, and unmanned assets that function robustly across the 
information and capability divides between platform-specific mounted and 
dismounted technologies. 

Fundamental Research 

Fundamental research was identified that will be required to enable the realization 
of future Soldier-autonomy team capabilities. Several of the most critical research 
questions cut across major capability themes identified previously: 

Human State Characterization: There is a need for characterizing the human state 
(cognitive/affective/social) in ways that are useful to and usable by (i.e., that can 
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be reasoned about) autonomy and other humans to facilitate mutually adaptive 
teams. Underlying this general requirement are research needs that include 
understanding variability across and within humans; understanding the limits and 
the opportunities for new mathematical methods to address real-world levels of 
complexity with potentially unstructured and incompletely characterized state 
space; and understanding the relationships between sensing (i.e., characteristics and 
performance properties under operational conditions) and human state estimation 
biases and variability across multiple modalities and scales. 

• Recommendation: Human State Characterization is directly in line with the 
research into “Estimating and Predicting Individual Soldier Capabilities, 
States and Behaviors” identified by the 2015 Individualizing Technology 
for Effective Teaming ASPSM. This area presents an enormous, pervasive 
challenge that is critical to a wide range of military and nonmilitary 
capabilities. The Army recently reprioritized significant internal and 
external fundamental research efforts (starting FY17) that are capitalizing 
on recent technological breakthroughs to enable significant progress in this 
area. The Army’s large collaborative effort across industry, academia, and 
the government should be continued. 

Enhanced Communication between Humans and Machines: There is a need to 
develop better approaches to exploit human multisensory capabilities and share 
concepts for effective communications between humans and machines. Underlying 
this requirement are research needs including cross-modal approaches to enabling 
real-time human comprehension under constraints of bandwidth, information 
quality, and task; predicting the effects of individual Soldier variability in sensory 
perception and utilization, as well as the effects of variability arising from 
operational/technological factors (i.e., clothing, headsets, crewstations) on the 
ability to comprehend information; and developing approaches to share concepts 
across the information and capability divides of platform specific manned, 
unmanned, and dismount technologies. 

• Recommendation: This area presents basic and applied research challenges 
critical to Soldier-autonomy and Soldier-Soldier teams. The Army has 
small-to-moderate funding related to fringes of this research area. Advances 
related to natural language processing, advanced computational approaches, 
and novel interface technologies may enable breakthroughs in effective 
communications. With additional resources, the Army has the opportunity 
to make fundamental advancements on the critical issue of sharing concepts 
across the information and capability divides of platform-specific manned, 
unmanned, and dismount technologies. 
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Linking Individuals to Team Behavior: There is a need to understand the mutual 
influence and adaptations between individual and team behavior. How does an 
individual impact team behavior and how does team behavior impact the 
individual? Teaming is not about boxes and functionality, it is about 
interdependence among the boxes. Accounting for interdependence requires 
observability, predictability, and directability. Perception and adaptability provide 
feedback among these 3 elements and define research requirements that include 
developing the interface between observability and predictability; understanding 
how a transaction between humans and intelligent systems occurs; developing tools 
that help identify parameters critical in collaborative behavior; developing tools to 
understand the psycho-physiology of intelligent systems and hybrid teams; and 
developing meaningful measures to assess team behavior. 

• Recommendation: Linking Individuals to Team Behavior is directly in line 
with the research into “Linking Individuals to Emergent Team Behavior” 
identified by the 2015 Individualizing Technology for Effective Teaming 
ASPSM. This area presents an emerging challenge as autonomy increases 
in intelligence and capability. The Army currently has insufficient 
fundamental resources dedicated to this area. Addressing this challenge, the 
Army needs additional resources to begin a large collaborative effort across 
industry, academia, and the government focused on modeling emergent 
team behaviors as a function of individual human and nonhuman members, 
and approaches to augment individual behaviors and functions to improve 
team performance. 

Robust Human-Autonomy Integration: There is a need to understand approaches to 
match human capabilities with the capabilities of rapidly advancing autonomies. 
Underlying this general requirement are research needs that include identifying the 
critical attributes of agents that would successfully integrate in the crew-automation 
environment; understanding human-autonomy integration in terms of recruitment 
and assembly of teams; understanding how to establish training protocols and 
potential removal from service or prevention of inclusion; developing novel 
approaches to integrating humans and autonomy focusing on nontraditional tasking 
and decision-making processes; devising principles to develop an adaptable user 
interface to facilitate tuning to individual variability (biases, proclivities, 
capabilities) and principles and ethics underlying the generation, evaluation, and 
refinement of adequate mission plans for distributed human-autonomy teams. 
There is a need to define the kind of assumptions that can and should be made 
considering long-term societal shifts in the relationship between humans and 
automations, taking into account anticipated perfusion of sensor, computational, 
and control technologies throughout daily life. 
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• Recommendation: Robust Human-Autonomy Integration is consistent with 
themes within the research into “Enabling Joint Human-Technology Team 
Capabilities and Performance” identified by the 2015 Individualizing 
Technology for Effective Teaming ASPSM. This area presents an emerging 
challenge as autonomy increases in intelligence and capability. The Army 
currently has insufficient fundamental resources directly dedicated to this 
area but has unique capabilities necessary to address this challenge. To do 
so, the Army needs additional fundamental resources focused on robust 
human-autonomy collaborative decision making and functionality, as well 
as fundamental and applied resources in intelligent crewstation 
technologies. 

Human-Autonomy Teaming Analysis: There is a need to develop the requirements 
for tools to help design and build human-machine systems. We have tools to 
analyze automation, and we have tools to analyze human performance; however, 
analyzing teams of humans and intelligent systems is not a linear combination of 
these 2 and the current methods and approaches to analyzing heterogeneous teams 
of humans and autonomous agents remain deficient. Underlying this requirement 
are research needs that include developing system engineering practice and models 
to 1) better integrate the human; 2) account for dynamics of work, team, 
individuals, and environment; 3) better account for the concept of time; 4) present 
alternatives critically; and 5) account for intelligent systems working 
collaboratively with humans. Further sufficient optimality criteria need to be 
defined and implemented with appropriate metrics for human-autonomy team 
success. 

• Recommendation: Human-Autonomy Teaming Analysis presents basic and 
applied research challenges critical to all stages of technology development 
from early concepts to test and evaluation. The Army has minimal funding 
targeting the advancement of human-autonomy team analysis methods and 
approaches. To address this critical emerging challenge area, the Army 
needs addition fundamental and early applied investment in advancing real¬ 
time, human-in-the-loop methods and approaches to interpreting Soldier 
and joint Soldier-autonomy performance, as well as novel approaches to 
collaboratively integrating intelligent analysis technologies with human 
subject matter expertise. 

3.2 Distributed Information Processing and Data Analytics 

The proliferation of distributed big data requires radical rethinking of the US 
Army’s approaches and technologies for intelligence gathering, processing, and 
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analysis. Rapidly growing volumes of data are gathered by military assets, 
nonmilitary sensors everywhere (e.g., security cameras, mobile phones), the 
Internet of Things, and social media sites pose new technical and operational 
challenges, threats, and opportunities for the Army. The volume, diversity, density, 
velocity, mobility, and distribution of the data will be potentially multiple orders of 
magnitude greater than in today’s world. Considering the recent re-emergence of 
artificial intelligence (AI), especially the successes in Machine Learning, the Army 
finds itself at a scientific inflection point for the development of game-changing 
capabilities, as reflected in the findings to follow. 

Adversarial and collaborative reasoning and machine learning require 
advances in ways to penetrate and characterize an enemy’s AI systems, to exploit 
enemy AI, and to improve and protect friendly AI. Important scientific problems 
yet to be solved in this space include the ability for one AI system to learn from 
another; the ability for an AI to validate that another AI is behaving as expected 
and not working outside its parameters, or maliciously; and the ability for an AI to 
move without detection for stealth penetration and learning. Research in these 
topics may focus on cyber and electromagnetic warfare, and will require serious 
attention to study of realistic adversarial activities and sophisticated red teaming. 

Avoiding vulnerability to adversarial deceptions will be particularly important 
in ill-structured problems typical of military environments. These environments are 
characterized by complexity, emergent and inconsistent behaviors, ambiguous 
cause and effect, and highly dynamic environments and behaviors. New approaches 
are needed to make machine learning and reasoning more resilient and resistant to 
deception, false facts, biases, and influences. It might be possible that exploiting 
the anticipated high degree of connectivity on a future battlefield, heavily saturated 
with sensors and intelligent machines/munitions, will provide opportunities for 
cross-validation of facts and inferences. This area of research is likely to be closely 
coupled with the one mentioned previously, with related requirements. 

Machine reasoning in effective interaction with humans (users, developers, 
maintainers, etc.) is critical for military environments. However, phenomena in 
human-machine systems are notoriously difficult to understand, predict, and 
mitigate. Approaches are needed to ensure that conclusions, indicators, and 
warnings provided by intelligent data processors are consistent with human 
decision-making needs and are not counterproductive with respect to human 
cognitive processes. Novel methods for validation and verification will need to 
emerge to ensure adequate quality of operations by such complex human-machine 
systems. A long-term research effort, as a close collaboration of Army and 
academia scientists, is likely to be required to make significant progress in this area. 
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Exploitation of open-source intelligence (OSINT) and nontraditional sources 

are critical as the amount of easily accessible, military-relevant information has 
grown exponentially with the proliferation of social media sites and applications. 
Traditional OSINT sources include, but are not limited to the following: social 
media, news sources, mass transit maps and schedules, and the dark web. 
Nontraditional OSINT sources can include data about power usage, emergency 
responders, and hospital activity; cameras; traffic patterns; and mobile phone usage. 
Approaches are needed to accurately identify the signal amongst the noise in the 
massive amount of information and data fusion to intelligently bring together 
various sources and types of data, as well as to combine OSINT with human 
intelligence (HUMINT) and other intelligence information. A research program in 
this area should consider lessons learned in recent efforts in soft-hard fusion. 

Recognizing emergent phenomenon and pattern of life sooner than human 
analysts is seen as especially important when dense urban areas or megacities 
become the operational environment. Analysts and Soldiers are increasingly 
dealing with ill-structured, highly unpredictable problems having 1) emergent 
behaviors and 2) complex relationships with freedom of action and interaction, 
ambiguous cause and effect, inconsistent behavior, and uncertainty. Current 
methods of intelligence analysis are unlikely to be effective. It is hypothesized that 
machine learning approaches, algorithms, and systems can help make sense of 
complex situations, recognize emergent phenomenon faster than any single or 
group of analysts/Soldiers could, objectively explore a greater number of 
hypotheses, many of which might have escaped human attention, and overcome 
confirmation bias. A research program in this area must address collecting large 
volumes of highly diverse, yet militarily relevant, data and to engage social and 
behavioral scientists. 

Ensuring analysts trust in how data are collected, entered, stored, accessed, 
and analyzed in the system requires a range of new approaches. Data must be 
vetted with regard to its accuracy. Systems must be able to maintain provenance 
information such that analysts can always trace information to its source, determine 
who entered it into the system, and see who accessed it along the way. Trust will 
be developed through the consistent demonstration and improvement of the 
effective exploitation of these data. Examples of such exploitations will include the 
use of social network analysis for social influence prediction and prevention; on- 
the-fly machine learning capabilities to develop common ground and ad hoc 
questioning; and especially, moving from analysis of past events to prediction. 
While ARL has a strong portfolio of trust-related research, it is a challenging area 
where progress will require time. 
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Portable artificial intelligence capabilities (AI in My Pocket) seems enabled by 
the dramatic and continuing increase in digital data generated from sensors 
(physical, electronic, and social) combined with a growth in computing power (both 
realized and on the near horizon). Such systems could provide Soldiers with near- 
instantaneous knowledge to make near-immediate sense of complex environments 
and situations. Research must be performed on approaches by which AI can 
replicate, complement, and improve the human brain for information processing, 
sense-making, and decision-making for military-relevant problems. Successful 
research efforts will require close partnering with academia and the Army training, 
doctrine, and experimentation community. 

Infrastructure, methods, standards, and tools are necessary to make advanced 
analytics rapid, reflective, and deployable. Flexible infrastructure environments 
with sufficient diversity are needed to accommodate a variety of applications, 
domains, and users. Challenges include 1) standardization of data and ground 
truths, analytic algorithms, software tools, performance metrics, and validations; 2) 
physical infrastructure that scales in terms of processing/computer power and 
classification levels; 3) operating infrastructure in terms of format/primitives, 
training, and testing environment (batch or streaming); and 4) validation and 
verification of Concept of Operations (CONOPS)/doctrines at different 
classification levels on adaptive learning and evolving complex systems. Partnering 
with industry and the Army experimentation community will be critical to 
successful research in these areas. 

3.3 Neural-Inspired Artificial Intelligence 

Technologies in AI have the potential for significant Army payoff. Advances in 
neuroscience are leading to understanding of neural structure, function, and 
networking, providing new insights into the functional components of intelligence 
and how they interact. Focusing AI research on key Army issues will leverage 
commercial advances and accelerate system development. This meeting brought 
together a unique mix of research leaders in neuroscience, brain modeling, vision, 
artificial neural networks, AI, cognitive computing, and circuits to identify critical 
research areas. 

Several critical Army aspects of AI are unlikely to be addressed by any commercial 
enterprise. Unlike commercial applications, tactical military applications are reliant 
on heterogeneous architectures across Army platforms, networks, sensors, and 
processors. Distributed operation is essential and must be resilient to attack, 
mobility, and network failure. Architectures need to be engineered to enable 
modularity and composability, incorporate autonomous agents, and human 


Approved for public release; distribution is unlimited. 

11 



interaction. They need to be linked with control and learning to enable active 
attention and problem solution in time-critical scenarios, and readily embedded into 
applications. They cannot be reliant on massive supervised training data sets but 
instead must incorporate prior knowledge. 

A key finding for the long term is the need for a multi-disciplinary combination of 
neuroscience and AI. There is a separation between computational neuroscience 
(brain modeling) and AI, with relatively little research overlap. Multi-disciplinary 
research should be enhanced, benefiting both Army AI and brain research. 

Army AI research should 1) focus on distributed AI for tactical application 
and 2) couple AI and action. 1) Distributed intelligence is not a current research 
emphasis in the AI research community, although some limited aspects might 
emerge commercially due to dense internetworking. Distributed intelligence is 
critical to Army applications and should be coupled with cognitive networking to 
provide resilient AI systems. Neuroscience is leading to understanding of neural 
structure, function, and networking within the brain, providing new insights into 
the functional components of intelligence and how they interact. Mathematical 
modeling is needed for this often empirically based area. 2) Active perception, 
focus of attention, motor control, and other cognitive programs are evident in 
neuroscience. These techniques should be explored for the combination of AI and 
robotic autonomy in dynamic Army settings (e.g., in task-based models). There are 
large fundamental gaps in understanding the use of memory and attention in AI. 
Recent neuroscience has demonstrated focus of attention mechanisms, for example, 
in vision. Long-term multidisciplinary study is needed to develop the science and 
practice of action-oriented AI. 

Beyond human sensing, AI should be applied to Army sensor fusion and 
processing. The recent success of artificial neural networks (ANNs) has 
demonstrated early progress in fusion of speech and vision. These techniques 
should be expanded upon and applied to Army problems (e.g., the fusion of 
electronic warfare, IR, and active sensing). Long-term research issues include 
fusion over different sensing and time scales, and coupling with action-oriented AI 
to enable autonomous mobile sensing platforms. 

A fundamental new science of Soldier-AI interaction is needed. Human- 
machine teaming is largely heuristic, and theoretical underpinnings are needed to 
develop systematic design methods. Psychological insight, mathematical modeling, 
and systems engineering must be combined and not left to separate research 
avenues. The science of interaction should encompass multimodality, including 
speech, vision, gestures, virtual reality, and brain-computer interfaces with 
wearable sensors such as electroencephalogram (EEG). AI designs must 
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incorporate the ability to query humans in 2-way dialog for learning, understanding, 
and task-oriented teaming. 

AI architectures are critical and fundamental design principles are lacking. 

Some key issues include the following. 1) Data abstractions and representations can 
determine the success of AI; however, this is an insufficient understanding and 
theoretical underpinning for this problem. 2) Incorporating prior knowledge is key 
to reduced training cycles and unsupervised learning. The limitations of big-data 
supervised training are now apparent and pose significant barriers to applying AI 
in Army scenarios. 3) Active learning is needed over different time scales with 
adaptive cost functions. Neuroscience demonstrates that learning is continuous and 
adapts and changes focus over time. A fundamental science of learning is needed 
that leads to algorithms capable of these more general learning attributes, coupling 
with control to enable active learning. 4) AI and memory should be coinvestigated. 
Human memory appears to be massively distributed, providing resilience and 
robustness. Long-term memory consolidation in humans appears to be a slow 
process, whose understanding is likely to provide new insights into AI architectures 
and processing. Neuroscience studies and knowledge bases should be 
coinvestigated to advance fundamental understanding and provide new memory 
architectures and algorithms, with tight coupling to active learning and distributed 
AI. 5) Links are needed between empirical ANN architectures and theoretical AI 
designs. Work in layered representations should be accelerated to provide 
systematic design principles and direct match with applications. 6) Circuit 
architectures are fundamental. Size, weight, and power (SWAP) constraints and 
reliance on wireless networks will continue to frame the achievable AI for Army 
scenarios. While commercial success of AI will drive circuit development for 
mobile applications, it is incumbent on the Army to discover canonical 
architectures that provide modularity and to move AI to embedded applications. It 
is also important to consider going beyond the classic von Neumann computing 
paradigm. 7) Data collection will continue to be important to support appropriate 
experiments in the long term. As AI systems emerge, they will become increasingly 
complex and multiplatform, creating challenges for large-scale experiments, 
including the need for costly research infrastructure. 

Army AI R&D would strongly benefit from a collaborative research alliance 
(CRA) or similar program with the above focus. Currently, in-house efforts 
largely consist of applying emerging AI tools in several disparate disciplines (e.g., 
autonomy, training, human-machine interaction) but are not broadly focused on 
long-term Army-relevant basic research in AI. Emphasis should be placed on the 
combinations of sciences needed, that go well beyond current collaborations. The 
merger of neuroscience, engineering, and computer science should be fostered and 
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may require new in-house organizational elements. Links with autonomy, 
intelligent agents, and robotics will bring leap-ahead potential to already strong in- 
house efforts. While a focus on tactical application is recommended, large-scale 
intelligence should also be explored in the context of distributed high-performance 
computing to bring in-house efforts up to the state of the art and beyond. A CRA 
will provide synergy and unification for the long term, enable government 
personnel development in this rapidly evolving area, and accelerate the application 
of AI technology to specific Army problems. 

3.4 Materials for Sustainable and Mission-Flexible Intelligent 
Systems 

We hypothesize that integrated innovation in materials, synthesis, and energy has a 
fundamental role to play in disruptively advancing desirable and necessary 
intelligent platform capabilities and behaviors over a range of moderate-to-extreme 
operating conditions. However, this hypothesis also maintains that advances in 
“classical'’ materials, synthesis, and energy may not be sufficiently capable of 
expanding beyond their own inherent limitations without new and meaningful 
connectivity to other—perhaps disparate—physics, chemistry, biology, and 
computation phenomena. 

Every meeting invitee was a noted pioneer in fields and activities related directly 
and indirectly to the meeting; indeed, it was the significance of their professional 
accomplishments to date that made them especially critical to helping define where 
the current gaps—and opportunities—lay in this area. The goal of this meeting was 
to create a conversation that would allow for materials, synthesis, and energy to be 
discussed explicitly and simultaneously in the unique context of intelligent systems. 
Mass, volume, and energy-efficient bounds defined the fertile space for identifying 
the knowledge gaps and transdiscipline work that needs to be done so that offsets 
in intelligent platform performance could be ensured deep into the Army’s future. 

The overarching conclusion was that, indeed, the potential performance benefits of 
intelligent systems and robotic devices are limited by a lack of comprehensive 
innovation and invention that more fundamentally addresses the scale-appropriate 
energy, power, and materials needs of these systems to deliver desired bulk 
responses, behaviors, and capabilities. There were many interesting and relevant 
research areas and thrusts identified with varying degrees of fidelity, and an attempt 
has been made to digest these into 5 key recommendations: 
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Material and Energy Intelligence in Robotic and Autonomous Systems 


The Army needs to invest in the science of scale-appropriate energy and power 
strategies unique to intelligent systems. The ASPSM participants consistently 
identified physical constraints associated with current energy and power 
approaches as a critical limitation on intelligent system performance and potential. 
A holistic, comprehensive, and more deeply integrated approach to energy is 
recommended and would include new research for enabling intelligent 
management, generation, harvesting, storage, conversion, and distributed use of 
energy and power. This includes introducing unprecedented levels of adaptivity so 
the intelligent system can develop new symbiotic and other types of energy 
acquisition relationships (e.g., foraging of natural and manmade materials, ambient 
energy harvesting) within its operating environment. It would also include the need 
for new concepts for materials and processes that could contain the reactive 
materials necessary for self-sustainment (e.g., flexible materials to contain micro¬ 
combustion, catalyst materials for synthetic metabolism to generate and consume 
energy-dense stored chemicals, and modes of planned material and structural 
degradation to enable “self-cannibalism” for extending operation under critical 
conditions). Research investment in these new concepts provides the underpinning 
science foundation for potential agent-agent teaming of different scale intelligent 
systems to improve mission endurance, durability, and environmental adaptability. 

Disrupt the Notion of an Intelligent System as a “Fixed Platform” 

The confluence of materials, synthesis, energy, sensing, and computation could 
provide breakthrough approaches to address current limitations to intelligent 
system performance, maneuver, adaptivity, endurance, and resilience. Materials 
and subscale functional elements could be designed to promote rapid assembly and 
synthesis of materials across multilength scales, including materials that can be 
rapidly disassembled and decomposed so they can be repurposed based on the 
global and evolving needs and task demands of the intelligent system. In addition, 
new research is advocated to provide the intelligent system the low-power ability 
to create and deploy tools to better perform its assigned tasks and enable its own 
self-sustainment for significant gains in operational endurance. Therefore, the 
Army needs to invest in materials research for reconfigurable electronics and 
materials, distributed actuation with localized power, and computationally driven 
point-of-need tool synthesis informed by task requirements and environmental 
constraints. 
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Aggressively Pursue Research into Multifield Metamaterials 


In recent years there have been burgeoning metamaterial demonstrations, with 
significant successes in complex custom electromagnetic materials for optics and 
antennas, and early demonstrations of passive mechanical metamaterials. Advances 
in microfabrication, synthesis, and additive manufacturing have matured to where 
we can start to realize complex multimaterial structures. This, combined with 
continued progress in simulation and reduced cost computation, means it is an 
opportune time to pursue research in multifield metamaterials with the goal of 
achieving novel materials that move us far beyond the limitations of traditional 
materials. For example, realizing scalable, tunable, highly efficient actuators 
supporting complex articulation, with potential for integrated sense and control, as 
well as materials for novel platform frames, with unique mechanical, optical, and 
electromagnetic properties. The successful realization of these broad potential 
materials will revolutionize autonomous platform design, performance, and 
endurance, for example, by enabling novel high-density energy sources to directly 
drive actuation. In addition, these materials have the potential to advance broader 
Army system performance included in the Cyber-Electromagnetic Activities 
(CEMA) communications and platform arenas. 

Cyber Synthesis for Making Good on the Promises of Multifunctionality, 
Bioinspiration, and Point-of-Use Adaptivity 

Currently, synthesis and fabrication technologies fail to enable much richer and 
resilient multiscale material functionality and contiguous, defect-free morphologies 
and subscale structures. Such processes also fail at achieving more efficient 
materials interfacing for discrete device/component integration. Therefore, the 
Army needs to invest in exquisite manufacturing of small building blocks and 
infrastructure with multifunctionality and point-of-use adaptivity unique to 
intelligent systems. For example, fundamental advances in synthesis could fully 
enable the benefits of a “circulatory system” to support distribution of alternative 
high-density energy and complex processing of energy at the location of 
consumption. The goal of this investment is to enable energy, power, actuation, and 
computation resource elements synthesis in a more distributed and resilient manner 
as needed in the global intelligent system. The benefits of such an investment would 
provide the scientific basis for informing the development of synthesis and 
fabrication methodologies to enable far more complex and energy-efficient robotic 
and intelligent system capabilities, responses, and behaviors. 
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Pioneering Opportunities for Computation and Modeling in Intelligent 
Systems 

New modes of embedded computation and sensing will be equally critical to 
enriching the potential of material and energy performance in intelligent systems. 
Recent and disruptive advances in theoretical and computational material and 
energy frameworks, including “materials by design”, and materials that can provide 
logic functions and sensing can enable “state machines” and disruptively shrink the 
response time and enable efficient new behaviors for maneuver and actuation by 
distributing control within the materials themselves. Similarly, advancing the 
development of AI as a tool to assess constraints and stretch the solution space for 
intelligent system design of materials, structures, processes, and energy and power 
cycles could enable leap-ahead performance gains in mass, volume, and energy 
efficiencies. 

3.5 Bioenabled Materials Synthesis and Assembly 

Recent advances in systems and synthetic biology, the associated high throughput 
screening methods, and genetic editing tools have allowed for holistic engineering 
of biology. Further, biological systems inherently produce spatially designated 
assemblies of complex hierarchical structures with extreme precision and have 
living material characteristics such as re-healing, adaptability, and dynamic, 
programmable properties. The meeting on Bioenabled Materials Synthesis and 
Assembly explored the fundamental questions and anticipated tools to command 
and control biology to achieve new materials and disruptive capabilities for the 
Army that are not available through traditional synthetic and processing methods, 
such as accessing material states not at thermodynamic equilibrium; multimaterial 
hierarchical assemblies with precise chemistry and architecture at the atomistic and 
molecular scales; and materials that dynamically respond to stimuli or changing 
conditions. Particular consideration was made for robustness of biological 
machinery and living products under nonaqueous and extreme conditions. Several 
capabilities were identified that may be enabled by biological approaches to 
synthesis and assembly. Understanding the biotic-abiotic interface and controlled 
assembly, disassembly, and reassembly could enable a very wide array of new, 
tunable, and responsive functionality and architectures. 

Utilizing a dynamic and interactive meeting format, several promising research 
opportunities were identified, down-selected, and further explored through an 
evolving series of small group discussions. The opportunities are discussed in the 
following paragraphs. 
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Design, Storage, and Implementation of Instructions for Materials Synthesis 
and Assembly 

Managing instructions for the synthesis and assembly process is a significant 
challenge and offers numerous opportunities for new research. A compelling 
desired capability is spatial and temporal control over assembly of the material 
architecture, which might include tracking the process via feedback in real time and 
using active control to steer it to a desired outcome. This is an intriguing goal that 
may demand synthetic methods and/or target materials that are stimuli responsive, 
reconfigurable, and switchable. Materials that are dynamic as part of their 
properties (e.g., exhibiting multiple metastable states or dynamic equilibrium) 
might further enable reorganization after a material is formed. For self-regulating 
synthetic systems, there is a need to tune equilibrium end-state for a continuum of 
products. A more passive approach is to provide cells the consolidated instructions 
for complete synthesis and assembly. There is much emphasis currently on top- 
level regulation with the opportunity to better develop control or modification of 
the proximal synthesis pathways. A worthwhile but challenging goal is breaking 
down hierarchical levels of cell function to manageable subunits to be used as 
“subroutines” for existing systems. Programmable control over active subunits may 
then be used to create designer materials. Self-organizing strategies are needed for 
complex/multicomponent systems, including self-assembly and self-repair of 
physical architecture and specified function across multiple scales. Programmed 
synthesis of intrinsically nondesignable systems is a particular challenge that 
highlights a theory gap—in particular, the inverse computational methods required 
to extrapolate from a targeted end-point back to the genetic origin and then generate 
the forward instructions for synthesis. Innovative strategies must also be considered 
to encode out-of-equilibrium higher organization and function. 

Robustness to Extreme Conditions Including pH, Temperature, Unstable 
Nutrient Supply and/or Nonaqueous Environments 

Living systems and their subcomponents are typically not robust to “extreme” 
environmental conditions such as temperature, pressure, and pH. Extending the 
operational range is a big challenge. Several ideas were considered, such as 
developing support systems that address specific challenges including biotic/abiotic 
vasculature that deliver nutrients and exoskeletons that shield the sensitive 
components from heat, pressure, or other stresses. Engineering biosystems for 
robustness was considered the most practical approach at this time. We may take 
advantage of the inherent adaptability of biological materials that enable survival 
under hostile conditions—for example, to adapt to changing energy sources 
(organism-level adaptation) or to develop antibiotic resistance (community-level 
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adaptation). Four fundamental categories of engineerable biological materials and 
specific pathways to engineering were discussed. Bioorganisms such as bacteria 
and yeast have highly efficient metabolic processes that enable low-energy 
operation and limited waste, and evolutionary adaptability to new conditions or to 
develop new capabilities including operation under extreme environments. 
Biomachineries such as mitochondria have scalable, fast, energy-efficient synthetic 
processes with high fidelity and error correction, and the ability to be abiotic, 
multienzyme, and cell-free. Biomacromolecules such as proteins and nucleic acids 
have native hierarchical structures, dynamic operation, and the ability to synthesize 
materials with high precision, selectivity, and multifunction. Biotic-abiotic hybrids 
are a more complex approach with challenges involving multicomponent 
interactions at various length scales that may be addressed through multiscale 
modeling and bioinformatics. 

Autonomous Sensing, Learning, and Evolving of Integrated Functions and 
Architecture 

Biological systems have an advantage over a synthetic assembly of materials in that 
they can respond to a huge range of stimuli, including physical/mechanical, 
chemical, optical, and electrical. However, controlled use of living materials 
requires new paradigms for instructing individual cells and their subcomponents as 
well as for organizing cells across wider morphologies. Coordination of 
multicellular systems in particular will require a greater understanding and control 
over signaling and communication. Biogeneration of nonliving products has the 
opportunity to address non-natural materials that cannot be synthesized through 
other methods. Directed evolution in particular has significant potential to address 
complex problems for which current chemistry and materials science approaches 
are inadequate, such as high-performance adhesives. Significant effort will be 
required to identify appropriate and potentially complex selection pressures to 
produce integrated functional and architectural properties. Integrated 
computational approaches should be included to accelerate learning and success. 
Biogeneration of materials with living biology included has the opportunity to more 
actively incorporate living characteristics in the product, such as sense and 
response, self-healing, and continually evolving properties. Research is needed to 
better understand mechanisms regarding inheritance of properties as well as how 
the living components can be controlled to be responsive or noninteractive on 
demand. Providing dynamically responsive or latent activity of the living 
component may enable environmental robustness and have unique application 
value. 
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Army Impact and Path Forward 

Bioenabled materials synthesis and assembly have the potential to provide access 
to sequence- and morphology-defined synthetic materials, with control from the 
micro- to molecular level. This unprecedented control may enable new concepts in 
structural and functional material design, and new modes to mitigate damage or 
failure. Suggested research topics that support the Army mission include the 
following: 

• Biomanufacturing Science and Rapid Engineering: Nature extensively uses 
biology to manufacture complex 3-dimensional structures of different 
shapes, sizes, properties, and functionality; however, this is a slow 
phenomenon that can take years to complete a given object. This research 
will develop an understanding of the principles that allow bioassembly and 
manufacturing of natural objects and identifying factors that could 
controllably accelerate the process to reduce maturation time. 

• Understanding the Biotic-Abiotic Interface: Fundamental theoretical and 
experimental studies to understand the interaction (bonding, adhesion, 
electron, and charge transfer) between biotic and abiotic materials. 
Knowledge gained from this research will impact all bioenabled research. 

• Bioenabled Structures for Harvest, Conversion, and Storage of Energy: This 
research will focus on assembling materials that can absorb sunlight or 
ambient chemicals, including waste, to produce and store energy parallel to 
natural photosynthetic and chemosynthetic processes. 

• Bioenabled Materials for Material Conversion: This research incorporates 
advanced recycling concepts to develop and couple biodegradation and 
biomanufacturing processes to convert natural resources and discarded 
synthetic materials to new products. 

• Bioenabled Materials for Signature Management: This research will focus 
on photo-, thermal-, and pressure-sensitive materials for real-time control 
over visibility, including Time-to-Live. 

• Bioenabled Information Sensing, Recording, Storage, and Retrieval: This 
research will use biological materials to generate, store, and communicate 
complex nonlinear data. 

There is considerable external investment in the rapidly developing field of 
biosynthesis, but the skills, expertise, and equipment required to carry out this work 
is typically unique, making it difficult for personnel from other disciplines to 
effectively collaborate with and leverage those efforts. One of the gaps highlighted 
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by the workshop attendees was the need for better communication between 
biologists, who better understand the capabilities available, and materials scientists, 
who better understand how to anticipate and meet application needs. One 
recommendation was for a series of workshops uniting these communities to 
increase exposure of cross-disciplinary opportunities and to resolve the biotic- 
abiotic language gap and different expectations of research timelines. These 
workshops may also help resolve how best to establish collaborative centers 
between the Army and other institutions. 

Additional resources will also need to address the research gaps to enable the 
opportunities in this field. Gaps specific to biology include reconciling the critical 
role of water in biological systems with engineered synthetic material systems, 
methods to stimulate forced adaptation and select successful candidates to achieve 
high-performance materials, and a library of primitive programmable biological 
subroutines and inverse computational strategies to achieve complex structure and 
function in inherently nondesignable systems. These needs will require significant 
effort, and as they apply to Army applications, they may best be met by investing 
in a critical mass of internal experts that can appropriately steer and leverage 
external efforts through collaboration and transition. There are also broad, 
underlying gaps that do not require specific expertise in biological materials but are 
just as critical. Primary among them are the need for fusing multiscale modeling 
with bioinformatics to accelerate the rate of discovery and understanding, and the 
need for characterization tools that address biological scales, environments, and 
material types with the resolution to probe interfacial interactions. The Army has 
established expertise in multiscale modeling and characterization that may be 
grown and tuned to the biological issues presented here to provide leadership in the 
field toward solving Army-relevant problems. 

3.6 Towards a Science of Complexity 

A new way of thinking is made necessary by the demands of contemporary science 
to overcome the complexity barriers to modeling complex systems in multiple 
scientific disciplines across the Army. For example, nondifferentiability of 
turbulent fluid flows, unpredictable transitions of peaceful demonstrations into 
riots, intermittent instability of decision making induced by physiological and 
psychological factors (i.e., emotion), inability to predict the behavior of large 
heterogeneous socio-technical systems, the catastrophic cascades of failure modes 
on power grids, computer networks, and so on. Complexity entails the qualitative, 
as well as the quantitative, richness of phenomena—a delicate balance between 
regularity and randomness in the nonlinear dynamics. 
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As a community, we know how to deal with algorithmic complexity, but an 
algorithm assumes that a problem formulation exists, which in turn means that a 
mathematical model exists. Today’s networks and systems are so complex that we 
do not have adequate mathematical tools to represent them, or even to talk about 
the complexity of representation. We need advances in mathematics to deal with 
this. Things are fairly simple if the world is heterogeneous, but well-mixed, and if 
we are primarily interested in steady-states. Often we do not have good spatial or 
temporal mixing, and even more frequently we are interested in transients (because 
external events occur before a steady state can be achieved). According to physicist 
Stephen Hawking, the 21st century will be the century of complexity. 2 

The goal of the meeting was to identify fundamental research issues that may enable 
future military-relevant capabilities and need to be addressed. Participants were 
asked to identify gaps in scientific understanding and describe how to apply 
existing scientific understanding to establish bounds on performance. 

The following questions provided a focus to initiate discussion: 

• What are the emerging mathematical, statistical, and methodological 
advances entailed by complexity? Assess and validate their suitability to 
perform complex adaptive analysis of scale-free dynamical systems in 
extreme hostile stochastic environments. 

• What are the fundamental algorithms and models needed to create 
autonomous systems that can characterize, detect, adapt to adversarial 
behaviors, and self-heal systems in a heterogeneous networked environment 
to enable effective situational awareness? 

• How do we measure, model, and predict the complex nonlinear dynamics 
of adjustment, adaptation, and intergroup levels that lead to emergent 
structure, culture, and physical processes (development of adversarial and 
allied groups, sociocultural conflict, emergent states, critical phase 
transition, etc.), which involves large-scale collectives, including 
organizations, crowds, societies, and cross-cultural relations that are critical 
to strategic planning and Army operations? 

• How can automated robust design, discovery, and decision be achieved 
through advances in optimal information exchange machine-learning/ 


2 Hawking S.‘Unified theory’ is getting closer, Hawking predicts. Interview in San Jose Mercury News; 2000 
Jan 23; 29A. 
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surrogate modeling, combined with physics, big data, uncertainty 
quantification, and optimal verification and validation? 

The mathematics that scientists employ does more than simply provide models of 
phenomena they seek to understand. It provides the language and logical tools 
necessary to think about those things in a way that enables them to make new and 
important predictions that can be experimentally tested. To “win in a complex 
world”, the Army must tackle problems wherein objectives and constraints evolve 
in unpredictable ways. Complexity arises from the increasing heterogeneity, 
connectivity, scale, dynamics, functionality, and interdependence of networked 
elements, from the increasing velocity and momentum of human interactions and 
information. Increasing integration of knowledge across disciplines requires a 
systems approach rather than a reductionist one. 

Consequently, the complexity of the connectedness problems being addressed by 
today’s society and Army challenges existing mathematics. To understand 
complexity science, mathematics must go beyond the analysis of analytic functions, 
not just in physics, but in the social, ecological, and life sciences as well. Fractional 
calculus is one way to frame the research hurdles entailed by complexity. Other 
approaches discussed include homotopy type theory (which combines algebraic 
topology, category theory, and logic), network and graph mathematics, as well as 
renormalization group theory. 

Based on conclusions drawn within the workshop, many of which overlap with 
those reached by the ARL’s ERA Gap analysis, we recommend the Army support 
internal, external, and collaborative programs in Complexity and Emergence to 
address the following 3 areas of critical need: 

Predicting Nonequilibrium Behavior 

• A mathematical definition of complexity and its quantification are lacking. 

• There is critical demand for mathematical models of finite size, far-from- 
equilibrium, complex multiscale systems, beyond statistical physics, and 
nonlinear dynamics. 

Lack of Physical Principles 

• We do not understand how the emergent dynamics and intelligence of 
heterogeneous collectives respond to externally imposed gradients or 
current densities. 

• Need to quantify sources and control of uncertainty and failures in complex 
systems. 
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Control of Information Exchange 

• Require theory for how information exchange controls automated robust 
design, discovery, and decisions that include big data, uncertainty 
quantification, failure, and optimal verification and validation. 

4. Recommendations 


Much of the discussion held at the meetings focused on the melding of dissimilar 
entities—for example, teams of humans and intelligent systems, biotic and abiotic 
materials, bulk materials, and functional materials. The recommendations that 
follow emphasize the need to address knowledge gaps that exist at the overlap 
between the different areas. These recommendations highlight a specific gap and 
also identify communities that should be involved in addressing the gap. Further, 
given the complexity of such heterogeneous systems, discussion across the 
meetings underscored the need for new analytic techniques to understand and 
predict the behavior of such systems. 

Intelligent Teams 

• Support a collaborative effort across industry, academia, and the Army 
research community focused on modeling emergent team behaviors as a 
function of individual human and nonhuman members, and approaches to 
augment individual behaviors and functions to improve team performance. 

• Support a multidisciplinary collaborative effort between the Army research 
community and academia that addresses combining neuroscience with AI. 
Research foci include distributed AI for tactical applications and coupling AI 
to action. 

• Support a multifaceted program (internal, external, and collaborative) in 
Adversarial and Collaborative Reasoning and Machine Learning to develop 
robust, friendly AI, and penetrate, characterize, and exploit adversarial AI. 
The research program must engage both social and behavioral scientists, as 
well as information and computational scientists. 

• Support a collaborative effort between industry, academia, the Army 
training and doctrine community, and the Army development community 
to develop portable AI capabilities (“AI in My Pocket”) that enable the near- 
instantaneous delivery of tailored context-specific support to Soldiers in 
complex environments and dynamic situations ranging from situational 
awareness and dynamic mission planning to embedded training and 
readiness. 
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• Support a collaborative effort across industry, academia, the Army research 
community, and the Army analysis community to establish an analytics 
infrastructure that addresses standards, methods, and tools for training, 
testing, validation, and verification of doctrine—for example, methods and 
approaches to interpret and quantify Soldier and joint Soldier-intelligent 
systems performance for physical tasks and for decision making. 

Implementation 

• Establish an academic center to develop the science of materials integration 
to enable the ingestion, circulation, and metabolic transpiration of energy and 
power for local actuation and computation in a distributed and resilient 
manner. 

• Support internal Army research efforts that focus on materials and subscale 
functional elements to promote rapid assembly and synthesis of materials 
across multilength scales, modeling and simulation, and fabrication 
technology to produce complex multimaterial multifield metamaterials. 

• Establish a collaborative effort between the Army research community and 
academia to develop fabrication and manufacturing processes and procedures 
to produce multifunctional building blocks unique to intelligent systems. 

Discovery 

• Support an extramural program in Complexity and Emergence to develop 
mathematical models for predicting nonequilibrium behavior of complex 
multiscale systems that advance our understanding beyond statistical 
physics and nonlinear dynamics. 

• Increase the hiring of mathematicians to enhance the capacity for 
mathematical analysis internal to Army laboratories. 

• Support a multifaceted program (internal, external, and collaborative) in 
Living Materials to develop the principles enabling bioassembly and 
manufacturing of complex synthetic and biocomposite materials and to 
identify factors capable of accelerating the process to reduce maturation time. 

• Support internal Army research efforts to enhance Army investments in 
multiscale modeling to accelerate the rate of discovery and understanding 
in biology by fusing multiscale modeling with bioinformatics. 
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5. Summary and Concluding Remarks 


The 6 meetings reflect those areas ARL considers of critical importance to the 
Army. The recommendations within each area are intended to show where 
investment will provide significant new understanding to advance capabilities 
desired by the Army. 

A theme that spanned the collection of meetings was the potential for increased 
capability if machine learning and AI were manifested across materials and within 
various length scales and not as an adjunct to an existing system. For example, co¬ 
locating computation with actuation in robotic systems benefits both local and 
global system response. Similar statements can be made about co-locating 
computation (intelligence) within manufacturing. 

A vulnerability discussed implicitly, if not explicitly, in several of the meetings is 
the need to maintain connectivity across disparate systems, some of which are fixed 
and others, mobile. Key to this is maintaining a consistent and coherent time base 
across a distributed and heterogeneous network of networks without resorting to 
global broadcast methods. This topic requires further investigation. 
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Appendix. Individual Meeting Summaries 


The meeting summaries found in this appendix appear in their original form, without editorial change. 
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Towards a Science of Complexity 


November 9-10, 2016 
Adelphi Laboratory Center MD 

Organizers: Dr. Bruce West (ARL/ARO) and Dr. Ananthram Swami 

(ARL/CISD) 

Introduction 

Organized by the Army Research Laboratory, the Army Science Planning and 
Strategy Meeting on "Towards a Science of Complexity" took place on November 
9-10, 2016 at the U.S. Army Research Laboratory (ARL) Adelphi Laboratory 
Center (ALC). A new way of thinking is made necessary by the demands of 
contemporary science to overcome the complexity barriers to modeling complex 
systems in multiple scientific disciplines across the Army. For example, non¬ 
differentiability of turbulent fluid flows, unpredictable transitions of peaceful 
demonstrations into riots, intermittent instability of decision making induced by 
physiological and psychological factors (such as emotion), inability to predict the 
behavior of large heterogeneous socio-technical systems, the catastrophic cascades 
of failure modes on power grids, computer networks, and so on. Complexity entails 
the qualitative, as well as, the quantitative richness of phenomena; a delicate 
balance between regularity and randomness in the nonlinear dynamics. 

As a community, we know how to deal with algorithmic complexity, but an 
algorithm assumes that a problem formulation exists, which in turn means that a 
mathematical model exists. Today's networks/systems are so complex that we do 
not have adequate mathematical tools to represent them, or even to talk about the 
complexity of representation. We need advances in mathematics to deal with this. 
Things are fairly simple if the world is heterogeneous, but well-mixed, and if we 
are primarily interested in steady-states. Often we do not have good spatial or 
temporal mixing, and even more frequently we are interested in transients (because 
external events occur before a steady state can be achieved). According to physicist 
Stephen Hawking, the 21st Century will be the century of complexity [1]. But that 
may already be here. 

The goal of the meeting was to identify fundamental research issues that need to be 
addressed, which may enable future military-relevant capabilities. Participants 
were asked to identify gaps in scientific understanding and describe how to apply 
existing scientific understanding to establish bounds on performance. The meeting 
encouraged structured yet open and broad-ranging discussion and exploration of 
multiple perspectives on the issues. 
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The following questions provided a focus to initiate discussion: 


• What are the emerging mathematical, statistical, and methodological 
advances entailed by complexity? Assess and validate their suitability to 
perform complex adaptive analysis of scale-free dynamical systems in 
extreme hostile stochastic environments. 

• What are the fundamental algorithms and models needed to create 
autonomous systems that can characterize, detect, adapt to adversarial 
behaviors, and self-heal systems in a heterogeneous networked environment 
to enable effective situational awareness? 

• How do we measure, model, and predict the complex nonlinear dynamics 
of adjustment, adaptation, and intergroup levels that lead to emergent 
structure, culture, and physical processes (such as development of 
adversarial and allied groups, sociocultural conflict, emergent states, critical 
phase transition, etc.), which involves large-scale collectives, including 
organizations, crowds, societies, and cross-cultural relations that are critical 
to strategic planning and Army operations? 

• How can automated robust design, discovery, and decision be achieved 
through advances in optimal information exchange machine - 
leaming/surrogate modeling, combined with physics, big data, uncertainty 
quantification and optimal verification and validation? 

The mathematics that scientists employ does more than simply provide models of 
phenomena they seek to understand. It provides the language and logical tools 
necessary to think about those things in a way that enables them to make new and 
important predictions that can be experimentally tested. The US Army’s Operating 
Concept is to Win in a Complex World [2], The Army must tackle wicked problems 
wherein objectives and constraints evolve in unpredictable ways. Complexity arises 
from the increasing heterogeneity, connectivity, scale, dynamics, functionality and 
interdependence of networked elements, from the increasing velocity and 
momentum of human interactions and information. Events now unfold in internet 
time, as noted by the: Defense Science Board (DSB) 2014 Study on Decisive Army 
Strategic and Expeditionary Maneuver. But complexity science is much more than 
network science. Increasing integration of knowledge across disciplines requires a 
systems approach, rather than a reductionist one. The DSB 2013 report on Resilient 
Military Systems and the Advanced Cyber Threat notes the exponential growth in 
complexity of operating systems software, and the software required to defend our 
systems. Increasing complexity in hardware and software points to challenges in 
testing them. 
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Consequently, the complexity of the connectedness problems, being addressed by 
today’s society and Army, challenges existing mathematics and often uncovers the 
tools lacking for an understanding of that complexity. The recognition of the 
importance of variability as a measure of complexity in the last decade or so, 
established that to understand complexity science, mathematics must go beyond the 
analysis of analytic functions, not just in physics, but in the social, ecological and 
life sciences, as well. Fractional calculus is one way to frame the research hurdles 
entailed by complexity to answer such questions as: How does a peaceful 
demonstration become a riot? Other approaches discussed include homotopy type 
theory (which combines algebraic topology, category theory and logic), network 
and graph mathematics, as well as, renormalization group theory. 

The following notes capture some of the discussion and findings of the meeting. 

Mathematics of complexity 

Complexity can be given an intuitive interpretation, but we acknowledge that there 
is no universally accepted definition. A quantitative measure of complexity can be 
associated with the dynamics of a system and is related to the number of interacting 
variables. Consider the dynamics of one or a few micro-variables, which is denoted 
“simple” in the lower left corner of Fig. 1. The mathematics of such systems, as the 
number of micro-variables increases, includes integrable and non-integrable 
Hamiltonians, nonlinear dynamics, linear control theory, algorithmic complexity, 
among other methodologies and are all relatively well understood. However, non¬ 
local effects in space and time are not well known even in the case of a few 
variables. As the number of micro-variables continues to increase, moving from 
left to right, a system's complexity rises and the known mathematical techniques 
become less and less useful. The methods blend into what we do not yet know. 
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(SIMPLE) Number of variables (SIMPLE) 


Figure 1: A conceptual nonlinear measure of complexity; one that categorizes networks with 
one or a few microscopic degrees of freedom and that are described by deterministic 
trajectories, as being simple. In the same way systems that have a very large number of 
microscopic degrees of freedom that are not described by individual trajectories, but rather 
by distribution functions, are also simple. Complexity lies between these two extremes of 
description [3]. 

On the far right side of the curve, where the number of micro-variables is very large, 
we have equilibrium thermodynamics, and the system is again relatively simple. 
The mathematics describing such systems involve partial differential equations that 
capture the evolution of probability density functions (PDFs), renormalization 
group (RG) relations and scaling of the coupling across scales; all of which bolster 
our understanding of the physical, social and life sciences. Here, the non-local 
effects in space and time are captured by the fractional calculus. As we ascend the 
curve, moving from right to left, the phenomena being analyzed increase in 
complexity, even though the number of micro-variables decreases, the number of 
macro-variables increases. Consequently, the mathematical tools available again 
become less useful. 

The unknown territory of future science and engineering lies between these two 
extremes of relative simplicity. The area of maximum complexity, at the apex of 
the curve, is where we know the least mathematically and have minimal control. It 
is where neither randomness nor determinism dominates; nonlinearity is 
everywhere; all interactions are nonlocal in space and time and some things are 
never completely forgotten. Here is where turbulence lurks, where the mysteries of 
neurophysiology take root, and the secrets of psycho-sociology of both large and 
small groups are hidden. All the problems in the physical, social and life sciences 
that have for centuries confounded the best scientific minds are here waiting for the 
next mathematical/scientific concept to provide some light of scientific 
understanding to guide the development of engineering control. 
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As a working definition, we may consider complex phenomena to have multiple 
interacting components, with emerging behavior that is entailed by, but cannot be 
immediately inferred from, the dynamics of its component parts, as would be the 
case using reductionist descriptions. The nonlinear interactions give rise to a blend 
of regular and erratic variability in complex phenomena, which enable them to 
adapt to a changing environment and thereby survive. The dynamics of complex 
nonlinear phenomena demands that we extend our horizons beyond analytic 
functions and classical analysis. 

The calculus of Newton and Leibniz and the analytic functions that solve the 
differential equations, resulting from Newton's force laws, have historically been 
seen as not only necessary, but sufficient as well, to provide a proper and complete 
description of the physical world. On the other hand, experiments indicate that a 
broad range of physical, biological and social phenomena cannot be understood 
using the analytic functions we have come to rely on in physics. These functions 
do not capture the complex dynamics of common physical phenomena such as 
earthquakes and hurricanes [4]; everyday social phenomena including group 
consensus [5], transitions from peaceful demonstrations into riots [6], economic 
unpredictably as in stock market crashes [7], high frequency finance [8] and 
healthcare networks [9]; or the familiar psychological activity of cognition and 
habituation [10]; in computer and communication networks such as Ethernet traffic 
[11], internet topology [12] and in the understanding of why networks fail [13]. As 
Perrow [13] notes, catastrophic failures may emerge because of unanticipated 
interactions in a complex system; mere redundancy may not be effective. The 
inherent complexity of these phenomena and many others is beyond the scope of 
familiar nineteenth century analysis, which, to a large extent, forms the 
mathematical foundation of present day physics and engineering. Understanding 
complexity as an extended class of problems, with common structural and 
mathematical properties, requires a new way of modeling and consequently more 
innovative thinking. 

Various books [14] [15] present a variety of strategies for modeling behavior such 
as entropy, fractal methods, power-law analysis, clustering, and the fractional 
calculus, to name a few. One approach being addressed is Hidden Markov 
Modeling (HMM) with non-negative matrix factorization for pattern learning [16], 
[17]. The general approach is being applied to both permissive environments (full 
observations) and is being tested for suitability for contested environments (sparse 
observations). The difference in the situations is the type, amount, and quality of 
data being assessed. Furthermore, a fundamental question being asked is related to 
the sampling method needed to learn a pattern. 
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Phenomena that require the notion of non-integer dimensions, derivatives and 
integrals for their interpretation were believed by most investigators to be 
interesting curiosities that lay outside mainstream science. However the increased 
sensitivity of experimental tools, the enhanced data processing techniques, the vast 
amounts of data made available by social media, and the ever increasing 
computational capabilities have all contributed to the expansion of science in such 
a way that those phenomena once thought to be outliers are now center stage. These 
curious processes are now described as exotic scaling phenomena, whose 
descriptions quantify the coupling of variations in phenomena across widely 
separated scales in both space and time. 

We note the lack of traditional dynamic equations in modeling complex 
phenomena, which have been explored using fractional thinking [15]. Such 
reasoning is a kind of in-between thinking; between the integer-order moments, 
such as the mean and variance, there are fractional moments required when 
empirical integer moments fail to converge; between the integer dimensions there 
are the fractal dimensions that are important when data have no characteristic scale 
length; and between the integer-valued operators that are local in space and time, 
are the non-integer operators necessary to describe dynamics that have long-time 
memory and spatial heterogeneity. Complex phenomena require this in-between 
way of thinking and the fractional calculus provides one framework for 
constructing this kind of mental map [15]. Fractional ARIMA processes have been 
used to model and analyze Ethernet traffic data, temperature and river level 
fluctuations [18]. 

A shift in thinking was accomplished by introducing probabilities to describe large 
complex system. Empirical exemplars of such complex phenomena are the time 
intervals between: earthquakes of a given magnitude [19]; solar flares of a given 
size [20], breathing intervals [21]; events recorded by electroencephalograms 
(EEGs) [22]; events for optimal storage in human memory [23]. Many, if not all, 
such empirical PDFs are inverse power law and the average time between events, 
the first moment, diverges. Thus, the evolution of the PDFs are no longer given by 
partial differential equations, instead they are controlled by fractional differential 
equations. 

It is important to have some perspective as to the ubiquity of phenomena whose 
statistics are inverse power law (IPL). They appear to be independent of context, 
occurring in geophysics, economics, sociology, medicine, astrophysics, computer 
and communication networks, urban growth, in short, in every scientific discipline 
from Anatomy to Zoology. The mathematics must therefore capture the complexity 
described by these IPL phenomena and answer such questions as: 
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. How can we translate a system’s complexity into a language that ultimately 
is easy to understand when presented to a human for decision making? 

• How to quantify the impact of complexity on system manageability, and 
human understanding and cognition? 

• Do we need one leader that understands the entire complex system to some 
extent? 

• Do natural complex systems have a design principle that can be learned? 

• Is there a dominant strategy for approaching phenomena that span 
disciplines? 

• How is uncertainty quenched by the flow of information? 

• How do we combine machine learning and modeling? 

• Are there fundamental limits to the complexity of engineered systems? 

• Should complexity be limited? Will simplicity lead to lack of robustness 
(anti-fragility)? Should (can) complexity be planned (engineered)? And 
what are appropriate frameworks for controlling complex systems? 

Modeling & Learning 

Autonomous systems must learn elements of the environment to perform well, 
which is coupled with supervised training from automation. Likewise, there are 
dynamic data driven applications systems (DDDAS) approaches [24] to deal with 
self-healing analysis such as assimilation methods. Key elements being pursued 
include graphical information fusion methods, advanced methods in nonlinear 
information fusion such as the Probability Hypothesis Density (PHD) estimation 
for unknown and varying number of targets, and incorporation of operating 
conditions to sense and model behavioral target variations. These elements are 
coupled with the information management approaches such as agent-based systems 
to compare information for a distributed shared situational awareness through 
situation assessment or even more challenging is shared situational understanding. 

Adversarial behaviors can manifest themselves in many ways; e.g., coordinated 
misinformation campaigns, campaigns possibly supported by social bots, 
impersonation, opinion manipulation, censorship by flooding, etc. The effect of 
adversarial behaviors could be direct (such as subverting a physical link or node in 
a communications network) or indirect (such as influencing opinion and trust 
relationships). Supervised and unsupervised approaches are required to detect these 
adversarial coordinated patterns. Are there fundamental limits to learning 
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adversarial behaviors? What are optimal approaches for rebuilding a network in the 
face of adversarial actions? Can autonomous agents implement optimal strategies 
based only on local information? 

The modeling of competitive sports has been used to provide insight into 
adversarial behavior, but one must exercise caution because competitive and 
adversarial behavior are not identical, although there is overlap. The subtle question 
relates to quantifying the difference between the two. The categorizing of agents as 
competitor or advisory often depends on the context and requires models having 
elements with itemized properties determined by a characterization space for the 
group of interest. Another difference is that game-theoretic formulations 
appropriate in this context could involve a very larger number of players, with 
heterogeneous, partially known, obfuscated and time-varying strategies. 

Dynamics of Complexity 

There are many nonlinear methods being explored for dynamical analysis. One 
such example is tracking, from which cubature (adaptive multidimensional 
integration of vector-valued integrands) methods are showing promise. The 
cubature points are adjusted to determine the uncertainty regions. While loosely 
aligned with emergent behavior, these types of approaches, when compared with 
other nonlinear methods, such as Homotopy (continuous deformation of one 
function into another) methods, are being used as a dynamic data driven application 
systems (DDDAS) methodology to update the models (models help for next state 
prediction). The challenge is determining whether or not these physics-based 
approaches are applicable to aspects of social-cultural-behavioral modeling. 

An example highlighting situational awareness is the combination of nonlinear 
movement predictions from cubature points combined with game-theoretic models 
of behavior. Thus, the system measures the nonlinear locations, uses a behavioral 
model, and then predicts the next course of expected action. While applied to single 
actors, it could be assumed that the behavioral model could also be aggregated to 
the actions of many actors that combined with intent, could be applied to command 
and control battle management. 

Nearly half a century ago the Physics Nobel Laureate Philip Anderson wrote the 
remarkable paper "More is Different" [25] in which he speculated on the collapse 
of reductionism in science, when seeking to understand phenomena of increasing 
size that encounter difficulties with scale and complexity. The central problem he 
identified is the shift from the quantitative to the qualitative, which he gathered 
under the heading of "broken symmetry". In physics symmetry means the existence 
of different viewpoints from which a system appears unchanged. 


Approved for public release; distribution is unlimited. 


35 



Newton appears to have been the first scientist to recognize the significance of 
symmetry in his assumption that space and time are homogeneous and isotropic, 
resulting in the law of universal gravitation being scale free. Symmetry is broken 
as the complexity of a system increases with size, the clearest example of which is 
the mathematically sharp, singular "phase transition" in which the microscopic 
symmetry (dynamics) is broken (violated) at criticality. Note that in the modern 
theory of complex networks, consensus in social networks shares most, if not all, 
of the properties of physical phase transitions, such as water becoming ice, 
including the spontaneous change of short-range interactions to long-range 
collective behavior. 

Phase transitions are observed as the consensus reached by social and neural 
networks, as a parameter that quantifies the interaction strength between the 
elements of the network is increased to a critical value. This is not unexpected for 
very large systems [26], but it came as a surprise when networks with as few as 9 
members display phase transition behavior in support of the concept of groupthink, 
in which a small group of individuals reach a unanimous decision that is the worst 
possible and would not had been reached by any individual within the group acting 
alone [27]. The implications of these formal results are still to be tested in terms of 
the efficacy of squads carrying out assigned tasks under adverse conditions. 

Formally, the micro-scale symmetry is lost to the emergent collective (less 
symmetric, or equivalently, more complex) behavior at criticality, which arises on 
the macro-scale and thereafter dominates the system’s dynamics. This behavior was 
successfully explained by another Nobel Prize winning physicist, Kenneth Wilson, 
with his invention of renormalization group theory (RGT) to describe critical phase 
transitions in physical systems. Here again the general theory shifts the emphasis 
from the quantitative to the qualitative to explain the origin of the loss of symmetry. 
In fact, RGT tells us that the macro-scale symmetry demands that a group as a 
whole responds to external forces, often in violation of the micro-scale interactions. 
This is not a settled question in physics, where the whole is not only more than, but 
very different from, the sum of its parts [25], and consequently one cannot yet 
predict what is to be learned by applying this new mathematical strategy to the 
social and life sciences. 

The difficulty in measuring complex phenomena can be traced to their lack of a 
characteristic scale, or said differently, their overabundance of interacting scales. 
RGT specifies how the various scales influence one another, with no one of them 
taking over the dynamics. Critical behavior can be anticipated by the existence of 
‘critical slowing down’ (CSD), which is the decay of a perturbation becoming 
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increasingly slower as a phase transition is approached. CSD might be the precursor 
to failure, which in itself is a qualitative change in the group dynamic. 

The notion of failure presupposes the existence of a model of a system’s 
functionality; a model that is either formal or informal. The most desirable model 
is quantified mathematically, since this is the simplest to unambiguously 
communicate scientifically and would be the most straightforward to verify 
experimentally. Of course, such a model is typically not available and most often 
the dominant variables with which to describe the system’s dynamics are unknown. 
Consequently, even when a potential model exists, it must be deduced from the 
data, for example, using Takens’ embedding theorem for attractor reconstruction 
[28]. But then, of course, one encounters the difficulties of data limitations and 
consistency. Another approach would be a universal machine learning model, 
which does not exist, and if it did exist would require criteria for determining when 
it had enough data. 

Interfacing Complex Models 

The discussion on robust design and optimal information exchanges focused on the 
interfaces between models at different scales / fidelity and the inherent problems 
fusing a system’s model from models of individual components. There are multiple 
challenges in describing, modeling and controlling heterogeneous systems that 
combine physical, information, and human elements, operating at different time 
scales, and represented by different mathematical and non-mathematical models. 
Users want tools and methods to reduce complexity, however only a few sets of 
studies have investigated physics-based and human-derived data aggregation for 
situation awareness [29]. 

The following gaps were identified: 

• Design-centric modeling (Translate uncertainty to human-understandable 
terms; e.g. when to evacuate from hurricane) 

• Information exchange: How do we communicate between different 
computer models/codes? Can we automate code refactoring (to automate 
generation of computer models of a complex system from models of its 
components) 

• Surrogate modeling that identifies low-dimensional manifold from high¬ 
dimensional manifold of parameters 

• Multiscale modeling: Space, time and physical domains (interrelated)— 
each has its challenges: methodologies needed 
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Machine learning from small data sets, and under adversarial conditions 


• Propagation and aggregation of uncertainty in complex systems; ability to 
deal with rare events 

Note: Much (most) of this is needed now, not 25 years out. 

Summary: A number of conclusions drawn within the workshop overlap with 

those reached by the ERA Gap analysis. 

Predicting non-equilibrium behavior 

• A formal mathematical definition of complexity and its quantification is 
lacking 

• There is critical demand for mathematical models of finite size, far-from- 
equilibrium, complex multi-scale systems, beyond statistical physics and 
nonlinear dynamics 

Lack of physical principles 

• We do not understand how the emergent dynamics and intelligence of 
heterogeneous collectives respond to externally imposed gradients or 
current densities. 

• Need to quantify sources and control of uncertainty and failures in complex 
systems 

Control of Information Exchange 

• Require theory for how information exchange controls automated robust 
design, discovery, and decisions that includes big data, uncertainty 
quantification, failure, and optimal verification and validation 
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Neural Inspired Artificial Intelligence 

November 29-30, 2016 
Aberdeen Proving Ground MD 

Organizers: Dr. Brian M. Sadler (ARL/VTD), Dr. Piotr Franaszczuk 
(ARL/HRED), and Dr. Raju Namburu (ARL/CISD) 

AI and Machine Learning Vision: The Army seeks to develop and employ a suite 
of artificial intelligence (AI) and machine learning techniques into systems to assist 
soldiers in complex operational conditions. These AI systems should be robust, 
scalable, and capable of learning and acting with varying levels of autonomy, to 
become integral components of networked sensors, knowledge bases, autonomous 
agents, and human teams. Potential far-reaching applications include discerning 
adversaries’ intent, supporting course of action analysis, providing real-time 
perception for rapid op-tempo autonomy, and developing intelligent adaptive 
soldier training. 

Objective & Scope: Many advances in AI have been informed and inspired by 
cognitive and computational neuroscience, which is leading to new computational 
algorithms and circuit architectures. The workshop objective is to determine new 
and long term research issues and areas to enable neural-inspired AI processing 
architectures that support the Army AI and Machine Learning Vision. 

The meeting brought together a unique mix of research leaders in neuroscience, 
brain imaging and computational modeling, vision, artificial neural networks, 
artificial intelligence, cognitive computing and circuits. 

Focus Areas: 

1. Computational brain modeling. Multi-disciplinary advances have led to 
enhanced understanding of neural processes. This session considers the trends and 
future of brain modeling with links to potential Army use. 

2. Brain-inspired architectures and processing. Inspired by advances in brain 
modeling, a variety of processing architectures have emerged, including single 
neuron models, deep neural networks, recursive neural networks, and others. These 
architectures have various tradeoffs and are typically tuned for a specific 
application. This session considers the future of architectures, such as modular 
composition and layered representations, and their implementation in HPC’s and 
dedicated devices. 

3. Data representations, training, learning, and applications. Data representation 
has emerged as a critical aspect of abstraction before processing, especially with 
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mixed modality inputs that can enable a general processing architecture to be 
effective for multiple applications. This session considers research issues 
associated with data abstraction, training and learning, as well as emerging and 
potential application areas for the Army. 

Background: 

Optimism with respect to AI and autonomous systems is due to recent advances in 
some key component areas. Technology convergence continues to unite 
networking, processing, sensing, and control onto portable devices and robotics. 
This follows general mass production trends in cellular networking, robotic, and 
sensor technologies. 

Artificial neural networks (ANN’s) have exploded into a variety of applications 
since 2010. While the basic ANN technology dates to the 1990’s, two trends have 
enabled their recent emergence. First, digital processing technology has advanced 
(i.e., Moore’s Law). ANN processing has been mapped onto graphical processing 
units (GPU’s) resulting in orders of magnitude speed-up. Second, training data sets 
are now available at the scale needed to ensure good statistical performance with 
brute-force gradient descent learning algorithms. These data may be on the order 
of millions of instances, each with ground truth to enable supervised learning, and 
are expensive to collect and organize. As big-data training became feasible, a large 
number of experiments have shown the empirical usefulness of ANN’s. 

Embraced by the largest US commercial enterprises such as Google and Facebook, 
ANN’s have been very successfully applied in areas such as image processing and 
vision, natural language processing, robotics, multi-agent systems, and others. They 
have displaced decades old technologies in image and speech processing. ANN’s 
are now better than humans on some kinds of visual object and word recognition, 
not to mention gaming. Recursive neural networks (RNN’s), also dating to the 
1990’s, enable sequence processing, such as sentences or video frames. Combined 
architectures can be used for fusion of modalities, e.g., imagery and audio are 
combined for the visual Q&A problem, where natural language processing is 
combined with vision, so that questions about the image may be posed to the 
system. 

Parallel advances in the mining of big-data to produce knowledge bases (KBs) has 
produced systematic methods for the storage, assimilation, and association of data, 
enabling rapid querying and information retrieval. Knowledge bases form the 
memory of intelligent systems such as IBM Watson, in game playing architectures, 
and numerous other applications. Machine learning (ML) can be thought of in this 
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vein, exploiting data to learn and specify models that can make predictions. ML is 
often viewed as a subset of AI. 

While these advances are significant and even dramatic, the long term question of 
intelligent systems and general AI for Army applications is reliant on fundamental 
long-term multidisciplinary research in areas such as neuroscience, signal 
processing, and computer science. 

The ANN, KB, and ML technologies mentioned above are geared to specific 
applications, whereas general AI seeks a much grander goal. Consequently, general 
AI is the subject of a great deal of fascination, speculation, and science fiction. 

Gaps and Recommendations: 

The interplay between neuroscience and computational architectures, AI, and 
machine learning has been fundamental to the advancement of AI. We lack a 
fundamental science of intelligent architectures that incorporate control (action), 
knowledge bases (memory), and human interaction. Consequently, AI architectures 
are typically built upon neural plausibility, and empirically driven for some specific 
application. Learning from biology has been key to AI, leading to the development 
of the theoretical model of ANN’s, and the combination of biology and artificial 
systems was common initially. However, over time the research split into two 
distinct approaches, one focused on improving neural models, and the other focused 
on application of early brain models to AI. Today there remains a large gap between 
neuroscience and computational neuroscience (brain modeling), and current AI 
architectures and their application. The multi-disciplinary combination of 
neuroscience and AI based in computer science and engineering remains relatively 
rare. This combination should be enhanced, benefiting both AI and brain neurology 
through interaction and cross-fertilization. 

A fundamental science of AI systems is needed that are: 

• Heterogeneous 

• Distributed / networked 

• Modular with composability, and map to efficient hardware 

• Readily embedded into applications 

• Not reliant on massive supervised training data sets, enable incorporation 
of prior knowledge 

• Have distributed memory 

• Can be updated, are adaptable and plastic, unlike current fixed ANNs 
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• Couple cognition, control, learning, and human teaming 

• Robust and resilient in combination with Army networking 

These broad goals contain several critical Army elements that are unlikely to be 
addressed by commercial enterprise. Tactical application is reliant on 
heterogeneous architectures across Army platforms, networks, sensors, and 
processors. Distributed operation is essential and must be resilient to attack, 
mobility, and network failure. 

Cognitive architectures should acquire the above features and be directed at 
specific Army applications. Many architectures have been proposed in the past few 
decades, but have only weak links with distributed application and other key Army 
aspects. These architectures should be engineered to enable modularity and 
composability, to incorporate autonomous agents, and human interaction. They 
should be linked with control and learning to enable active attention and problem 
solution in time critical tactical scenarios. 

Data abstractions and representations are key to the success of AI. A great deal of 
evidence shows that AI architectures may succeed or fail depending on whether an 
appropriate representation and dimensionality reduction approach is applied. 
However, there is insufficient understanding and theoretical underpinning for this 
problem. A continued research emphasis on dimensionality reduction methods, 
new measures of information, and their interplay should help. 

Links are needed between ANN architectures and theoretical AI designs. While 
ANNs have advanced empirically, the theoretical underpinnings are lagging. Early 
work in layered representations should be accelerated, to provide systematic design 
principles and direct match with applications, incorporating tools such as graph 
signal processing and nonlinear signal decompositions. 

Incorporating prior knowledge is key to reduced training cycles and use of 
unsupervised learning. The limitations of big-data supervised training are now 
apparent, and pose significant barriers to rapid application of AI in Army scenarios. 
It is critical that analytical frameworks be found that enable the incorporation of 
prior knowledge, such as the combination of Bayesian methods and other AI 
techniques, along with the study of development and learning in biological neural 
networks. 

AI and action should be coupled. Active perception, focus of attention, motor 
control, and other cognitive programs are evident in neurology. These techniques 
should be explored and developed for the combination of AI and robotic autonomy 
in dynamic Army settings, e.g., in task based models. There are large fundamental 
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gaps in understanding of the use of memory and attention in AI. Recent 
neuroscience has demonstrated focus of attention mechanisms, e.g., in vision, 
varying in time over spatial and resolution scales. Long term multi-disciplinary 
study is needed to develop the science and practice of action-oriented AI. 

Beyond human sensing: AI should be applied for Army sensor fusion and 
processing. The recent success of ANNs has demonstrated early progress in fusion 
of audio and vision. These techniques should be expanded upon and applied to 
Army problems, e.g., the fusion of EW, IR, and active sensing. Long term research 
issues include fusion over different sensing and time scales, and coupling with 
action-oriented AI to enable autonomous mobile sensing platforms. 

Distributed AI is needed for tactical application. Distributed intelligence is not a 
current research emphasis in the AI research community, although some limited 
aspects might emerge commercially due to dense internetworking and off-platform 
computing. Distributed intelligence is critical to Army applications, and should be 
coupled with cognitive networking to provide resilient AI systems. Advances in 
neuroscience are leading to understanding of neural structure, function, and 
networking within the brain, providing new insights into the functional components 
of intelligence and how they interact. Emerging understanding in network science, 
graph signal processing, and other new mathematical tools may bring mathematical 
modeling to an often empirically based field. 

Active learning is needed over different time scales with adaptive cost functions. 
Neuroscience now provides preliminary understanding for how learning is 
continuous and adapts and changes focus over time. A fundamental science of 
learning is needed that leads to algorithms capable of these more general learning 
attributes, and that couples with control to enable active learning. 

Aland memory should be co-investigated. Human memory appears to be massively 
distributed, providing resilience and robustness. Long term memory consolidation 
in humans appears to be a slow process, whose understanding is likely to provide 
new insights into AI architectures and processing. Neuroscience studies and 
knowledge bases and other computer science memory constructs should be co¬ 
investigated to advance fundamental understanding and provide new memory 
architectures and algorithms, with tight coupling to active learning and distributed 
AI. 

A fundamental new science of human-AI interaction is needed. Human-machine 
teaming is largely heuristic, and theoretical underpinnings are needed to develop 
systematic design methods. Psychological insight, mathematical modeling, and 
systems engineering must be combined and not left to separate research avenues. 
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The science of interaction should encompass multi-modality, including speech, 
vision, gestures, virtual reality, and brain-computer interfaces with wearable 
sensors such as EEG. AI designs must incorporate the ability to query humans in 
two-way dialog, for learning, understanding, and task oriented teaming. 

Circuit architectures are critical. SWAP constraints and reliance on wireless 
networks will continue to frame the achievable AI for Army scenarios. While 
commercial success of AI will drive circuit development for mobile applications, it 
is incumbent on the Army to discover canonical architectures that provide 
modularity, to enable dedicated circuit designs that have broad Army applicability. 
While this paradigm has been successful in Army applications such as cognitive 
radio, we currently lack a science of AI modularity and architecture composability. 
Tools are also needed to move AI algorithms to embedded applications. Long term, 
it is important to consider going beyond the classic Von Neumann computing 
paradigm for some AI components. 

Data collection and large scale experiments should focus on Army relevant 
scenarios. Data collection will continue to be important to support appropriate 
experiments in the long term. As AI systems emerge, they will become increasingly 
complex and multi-platform, creating challenges for large scale experiments, 
including the need for costly research infrastructure. 

Conclusion: 

The development and combination of AI technologies has the potential for 
significant payoff in future Army systems. Advances in neuroscience are leading 
to understanding of neural structure, function, and networking, providing new 
insights into the functional components of intelligence and how they interact. 
Focusing AI research on the key Army issues will leverage commercial advances 
and accelerate system development, with the strong potential for leap ahead 
capability. 

Acknowledgement: The ARL organizers gratefully acknowledge the substantial 
contributions of many ARL, academic, and industry colleagues. 
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Bioenabled Materials Synthesis and Assembly 

December 1-2, 2016 
Aberdeen Proving Ground MD 

Organizers: Dr. James Snyder (ARL/WMRD), Dr. Shashi Kama (ARL/WMRD) 

Co-Organizers: Dr. James Sumner (ARL/SEDD), Dr. Josh Orlicki 

(ARL/WMRD) 

Introduction: 

Biotechnology is a fast growing sector of the economy. Recent advances in systems 
and synthetic biology, the associated high throughput screening methods, and 
genetic editing tools have allowed for holistic engineering of biology. This may 
enable point of need and lower cost generation of chemical feedstocks and 
materials. Further, biological systems inherently produce spatially designated 
assemblies of complex hierarchical structures with extreme precision. 
Biotechnology may uniquely provide the DoD novel high-performance materials 
not accessible through traditional synthetic and manufacturing approaches and 
offers the potential to impart disruptive living material characteristics such as re¬ 
healing, adaptability, and dynamic, programmable properties. 

The Bioenabled Materials Synthesis and Assembly ASPSM gathered thirty 
participants, including top university professors from across the country, to explore 
the fundamental questions and anticipated tools to command and control biology to 
achieve new materials and disruptive capabilities for the Army that are not 
available through traditional synthetic and processing methods , such as non- 
thermodynamically accessible material states; multimaterial hierarchical 
assemblies with precise chemistry and architecture at the atomistic and molecular 
scales; and materials that dynamically respond to stimuli with desired response 
through changing conditions. Opportunities were discussed for coupling dynamic 
synthesis with precision assembly to define bold new approaches to material 
development that exploits unique benefits of biology. In particular, the unique 
structure/function relationships, error/fault tolerances, robust and diverse stimuli 
responses, and dynamic interfaces that abound in biological systems are of great 
interest. Consideration was made for relevance to all material categories as well as 
heterogeneous systems, with particular interest in robustness of biological 
machinery and living products under non-aqueous and extreme conditions. 

The workshop structure was modeled on that of last year’s Microscale Adaptability 
ASPSM, which utilized a non-traditional dynamic format emphasizing participant 
input to develop the focus areas of the workshop. Academic participants presented 
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concise 3 minute summaries of their research as it relates to the topic, which 
provided the group a comprehensive and diverse perspective of the current state- 
of-the-art in synthetic biology, biotemplating and related materials science. 
Participants were then organized into small groups to identify long-term scientific 
opportunities for coupling dynamic synthesis with precision assembly across all 
material categories and to define bold new approaches to material development that 
exploit unique benefits of biology. Subsequent breakout sessions addressed 
research pathways and critical scientific breakthroughs required to achieve those 
opportunities, particularly in conditions austere or extreme for biological function, 
and anticipated technological capabilities enabled by this research. The 
composition of the small groups was reshuffled for each breakout session to allow 
for different pairings of ideas and styles as the workshop progressed. The rest of 
this report details output from the workshop as a whole within the categories of 
opportunities, pathways, challenges and resulting capabilities. 

Recommendations and Opportunities: 

Long-term opportunities 

State-of-the-Art (SOA) Technologies in synthetic polymer synthesis and assembly 
via chemistry can generate polymers with large diversity, but cannot be 
implemented in a template-directed manner. Living polymerizations can yield 
some block-level and statistical control over monomer sequence that can extend to 
some control over polymer architecture through branching units and sidechains but 
lack information control in the monomer sequence. Solid state synthetic approaches 
yield oligomeric materials with defined sequence, but lack the error correction 
mechanisms and are limited in terms of accessible scale and compatible 
chemistries. At the next scale, nanostructured functional materials based on 
polymers can yield hierarchical assembly to some extent but are limited in control 
over system dynamics and very limited in control over system integration. 

Biological organisms have several potentially useful characteristics to consider as 
a basis for designing new approaches to material synthesis and assembly that 
address the deficiencies noted above. Organisms can reproducibly assemble 
precise, scalable, template-directed products of protein polymers, although they 
are generally constrained to the 20 canonical amino acids with limited expansion 
to post-translational substitution or unnatural monomers at this time. Organisms 
can sense or detect even minimal changes to their environment regarding light, 
temperature, pH, pressure, metabolite concentration, etc. Organisms can then 
generate a chemical, physical or electrical response to maintain operational 
equilibrium or performance, known as physiological adaptation. Communities of 
organisms may also collectively respond over time by evolving to maintain 
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optimized performance under changing conditions , known as genetic adaptation. 
Through these forms of adaptation, organisms can operate with energy efficiency 
and through use of environmental energy sources and local feedstocks. Organisms 
can also generate, store and transmit complex multivariate information. Multiple 
organisms can then demonstrate complex functions through collective and 
coordinated active behavior. Finally, all of these functions may be augmented 
through direct human intervention as organisms are mutable and can be modified 
for target properties e.g. through selective “breeding”, through forced evolution and 
through direct modification (e.g. DNA sequencing). 

Opportunities and challenges in using biology towards synthesis and assembly of 
materials may be very different depending on how biology is being used. 
Approaches may range from bioinspired, in which design lessons learned from 
nature are replicated using conventional chemical or mechanical synthetic 
approaches; to bioderived, in which material is harvested from biological 
organisms then used in a synthetic material; to biosynthesized, in which biological 
organisms are used as the direct mechanism of synthesis and assembly of the target 
material. While the intention of the workshop was to focus predominantly on 
biosynthesis, relevant opportunities within the other methods were also considered. 
In addition, consideration was made for the recent emergence of “living materials”, 
a hybridization of living organisms and nonliving materials that has significant 
overlap with biosynthesis and can include many target materials that are still living 
even in functional applications. 

The groups identified several potential advantages of using biological organisms 
for synthesis vs. traditional methods: 

• Diversity: Harnessing and expanding the rich palette of biomolecules (e.g. 
DNA, RNA, proteins, orthogonal polymers, etc.) and biomanufactured 
materials to go beyond petrochemical building blocks and traditional 
chemistry. Chemical security and improved domestic manufacturing 
capability are additional benefits. 

• On-demand: Biomanufacturing may be turned on or off, and rate controlled, 
through control over metabolite concentration or environmental conditions 

• Efficiency: Biological entities can be very energy and feedstock efficient 
during synthetic processes. As the technologies discussed here mature, there 
is promise for low cost of production. 

• Spatial control: Biomanufacturing may yield dynamic control over three- 
dimensional material architecture 
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• Temporal control: Synthesized output can be changed through changes in 
metabolite or environmental conditions 

• Fast adaptation: Bacteria, viruses and other rapidly evolving organisms may 
be used to adapt synthetic output to changing conditions 

• Information-based robust materials: Evolve “better” and chemically diverse 
biomolecules (e.g. DNA, polypeptides) for 

o Orchestrated control over structure and motion that adapts and evolves 
in 3D, which may require additional foundational development of the 
requisite chemically dynamical linkages to enable these adaptive 
characteristics. 

o Coordinated behavior (e.g. sensing, processing and response 
characteristics) of collective materials. 

o Information-encoded polymers based on complex hierarchical 
functional structures. 

• Bio-based materials provide opportunity for self-limiting and self- 
replicating control over information flow and energy flow in a closed 
system. 

Considering these factors, the academic participants provided significant input 
regarding the long-term research opportunities and challenges within the scope of 
bio-enabled manufacturing. This input brought forth and clarified the following 
three critical topics: 

1. How do we design, store and implement instructions for materials synthesis and 
assembly? Cells are a coexisting blend of precision (deep energy wells) & 
disorder (dynamic response/variability). 

a. How do we provide the consolidated instructions for complete synthesis and 
assembly? Working with current biological building blocks this could 
include for example DNA, peptide based assembly, or carbohydrates. 

b. How do we enable biosynthesized materials to inherit properties and 
advantages of the parent biology such as sensing, adaptation, switchable 
operation, and self-healing? Properties such as these may be critical for 
interfacing materials to living systems such as the human body. 

c. How do we enable spatial and temporal control over assembly of the 
material architecture? This capability might include tracking the process 
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via feedback in real time and using active control to steer it to a desired 
outcome. The phrase “dynamic orchestration” was considered. 

d. How do we provide instructions to encode out-of-equilibrium higher 
organization and function? Particular value may be placed on dynamic 
systems with active energy transduction. 

2. How do we make living synthesis / on-demand biomanufacturing robust to 
extreme conditions including pH, temperature, non-aqueous environments? 

3. How do we construct materials with autonomous sensing, learning and 
evolving of integrated functions and/or architecture? 

Pathways towards meeting the identified opportunities 

1. How do you design, store and implement instructions for materials synthesis 
and assembly? How do we consolidate those instructions and enable 
inheritance of properties, spatial and temporal control over assembly, and out- 
of-equilibrium higher organization and function? 

Cells have the ability to design and implement instructions for synthesis. There is 
a lot of emphasis currently on top-level regulation. Intermediate changes to a cell 
may be possible by changing RNA and related molecules. Viruses use this approach 
to assault cells by knocking out the internal genome and re-writing them to achieve 
a minimal & efficient function. There is less emphasis on the proximal pathways 
that actually create materials via active sub-units/sequences. This latter approach is 
worth considering. Large parts of the genetic code may be involved in logic 
mechanisms and provide a basis for implementing instructions for synthesis and 
assembly. A worthwhile but challenging goal is breaking down hierarchical levels 
of cell function to manageable sub-units. Analogous to circuits, it may be possible 
for cells and their sub-functions to be used as “subroutines” for existing systems. 
Programmable control over active sub-units may then be used to create designer 
materials. The enabling capability would be the implementation of multiple 
synergistic metabolic pathways in a single cell, or the coordination of multiple cell 
types in a system from a designed starting point. 

The information required to develop biological subroutines, and to further encode 
and sequence those subroutines to make a material, is unknown for many organisms 
and must be discovered. A suggested initial approach is to study organismal 
systems that possess the properties of interest to understand organismal growth of 
structures and underlying design rules. Examples include organismal production of 
magnetic nanoparticles, for which the mechanism is well defined; production of 
photonic crystals to exhibit a wide range of colors e.g. in butterfly wings and 
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peacock feathers; production of pigments such as melanin and carotene; and 
production of structural materials such as chitin and cellulose. The target of this 
discovery and design process would be a library of primitive functions that may be 
used to develop programmable biological subroutines pertinent to material 
synthesis. 

Applicable use of programmable biological subroutines further requires 
understanding of how to string the primitive functions together to achieve a desired 
chemical architecture. There is a need here to harness self-organizing strategies for 
complex/multi-component systems. This is a complex problem that must 
accommodate self-assembly of physical architecture and specified function across 
multiple scales, preferably with built-in feedback mechanisms to guide the process 
and undertake repair as necessary. The library of subroutines may be considered as 
“genes” for bio-enabled materials synthesis. To execute this type of research there 
is a critical need for tools to design genome complexity. We are currently able to 
program molecular level order of DNA scaffolding. It is less clear how we harness 
intrinsically non-designable systems. This need highlights a theory gap. Current 
capability allows us to go from a genetic starting point to predict the grown system, 
but we lack the understanding to extrapolate back from a targeted end-point back 
to the genetic origin. Cellular automata-type approaches that define interactions and 
watch a system grow are not appropriate here, it is the inverse problem that needs 
to be solved. The first step will be defining the fundamental capability that allows 
us to solve the problem using computational methods. It is helpful to consider 
analogous systems. In origami the fundamental capability is a fold, and it is now 
possible via computational origami to draw any final outcome and then derive the 
fold instructions. Application of this concept to biomaterials led to DNA origami, 
which further experienced exponential growth in the design space when non- 
symmetric folding was enabled. Biomimetic approaches to new materials might 
consider further exploring this concept with PNA or LNA in the near-term. In 
biomanufactured systems, challenges arise from vast energy landscapes with 
multiple energy wells. Chaperones or analogous approaches may be necessary to 
promote folding or other fundamental capabilities. 

Spatial and temporal control over assembly is an intriguing problem that may 
demand synthetic methods and/or target materials that are stimuli responsive, re- 
configurable and switchable. Biosynthetic systems can be responsive to multiple 
stimuli that steer microbes into desired functions or architectures. One route may 
be to use 3D printing to grossly arrange cells. Further combining light activation 
with 3D printing may yield orthogonal/complimentary spatial/temporal control. 
Engineering materials that are dynamic as part of their properties (dynamic 
equilibrium) might further enable reorganization after a material is formed. For 
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self-regulating synthetic systems there is a need to be able to tune equilibrium end- 
state for a continuum of products. 


2. How do you make living synthesis / on-demand biomanufacturing robust to 
extreme conditions including pH, temperature, non-aqueous environments? 

Living systems and their subcomponents are typically not robust to “extreme” 
environmental conditions such as temperature, pressure and pH. Extending the 
operational range is a big challenge. For nonliving targets and availability of a more 
controlled synthetic environment, bioproduction followed by removal or 
conversion of the living scaffold can sidestep operational constraints. Once the 
target material is constructed it will have same properties and environmental 
robustness, as any solid-state material. However, these approaches do not address 
the fundamental intent here to develop more robust living processes. One approach 
may be to develop support systems that address specific challenges, for example 
bio/abiotic vasculature that deliver nutrients, or exoskeletons that shield the 
sensitive components from heat, pressure, or other stresses. 

Engineered biosystems may be a better overall approach, such as enabling synthesis 
through control of the metabolic machinery of a cell, which may be on-demand 
based on environmental conditions or triggers. Traditional requirements to culture 
organisms to engineer them may no longer be necessary given recent developments 
for delivering DNA into cells in situ. In addition, some robust platforms already 
exist as a starting point. For example, black fungi found in Antarctica and subjected 
to experiments on the International Space Station are renowned for their ability to 
live under the harshest of conditions. Eukaryotes, including fungi, are amenable to 
genomic engineering, but only a few modules are available. The pathways of 
eukaryotes are not well explored, especially thermophilic eukaryotes. 

Similar to the previous section, it is suggested that the problem of “hardening” 
biological functions and mechanisms be addressed through development and 
design of fundamental building blocks that are designed to interface with each 
other. Many pathways work with the inherent adaptability behaviors of biology that 
already enable survival under hostile conditions, for example to develop antibiotic 
resistance. Four suggested categories with pathways to improved behaviors are: 

• Microbes (e.g. bacteria): Advantages include highly efficient metabolic 
processes that enable low energy operation and limited waste, and 
evolutionary adaptability to new conditions or to develop new capabilities 
including operation under extreme environments. Pathways to better 
engineering: 
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o Improving culturability through a focus on microbes in the mixtures, 
growth media and nutrient needs, including isolated systems and co¬ 
culture 

o Developing genetic tools to interrogate or modify isolated systems and 
that apply to non-model systems 

o Identifying appropriate bacteria to develop target capabilities or survival 
skills 

o Manipulating inter-cellular communication by interfacing with other 
engineered nanomaterials that provide cues 

• Biomachineries (e.g. ribosomes, mitochondria): Advantages include 
scalable, fast, energy efficient synthetic processes with high fidelity and 
error correction, and ability to be abiotic, multi-enzyme and cell-free. 
Pathways to better engineering include: 

o Enabling operation in non-biological environments 

o Understanding assembly and motion including system dynamics, multi- 
component assembly 

o Reconstituting the native function in non-native state 

o Isolating biological parts to reconstitute and re-assemble them into 
functional elements 

o Engineering biotic-abiotic interfaces 

• Biomacromolecules (e.g. proteins): Advantages include native hierarchical 
structures, chemical diversity available, monodispersity, dynamic 
operation, ability to synthesize nanomaterials and other structures with high 
precision and selectivity, ability to combine various building blocks toward 
multi-functional materials. Pathways to better engineering include: 

o Understanding structure-functional relationships in extremophiles 

o Developing enzymatic pathways under non-biological conditions 

o Understanding protein folding and enzyme evolution to function 

o Developing the biotic/abiotic co-assembly 

• Biotic/abiotic hybrids: A complex approach with challenges involving 
multi-component interactions at various lengthscale that may be addressed 
through multi-scale modeling and informatics 
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3. How do you construct materials with autonomous sensing, learning and 
evolving of integrated functions and/or architecture? 

Biology has an advantage in that it is more than just an assembly of materials. 
Biology can respond to a huge range of stimuli including physical/mechanical, 
chemical, optical and electrical. However, controlled use of living materials 
requires new paradigms for instructing individual cells and their subcomponents, 
including the methods suggested in the sections above, as well as for organizing 
cells across wider morphologies. Coordination of multi-cellular systems in 
particular will require greater understanding and control over signaling and 
communication. Three classes of materials were considered: 

i. The first class includes non-living products, which may be defined here as 
materials generated by biology but there is nothing alive in them after production 
or in use. Preferably, these are materials that are biologically produced but don’t 
exist in nature and cannot be synthesized through other methods. Complex and high 
performance materials will not be readily achieved using combinatorial or 
biopanning techniques. Directed evolution is a compelling approach that provides 
significant potential to address complex problems for which current chemistry and 
materials science approaches are inadequate, such as high performance adhesives. 
It may also be used to develop research capabilities, such as designing screens or 
selection for cells with certain material properties, or evolving cells as nucleation 
centers or organizational scaffolds. However, successful application of directed 
evolution approaches to optimizing materials properties will require new selection 
pressures to produce integrated functional and architectural properties. How to 
select for materials properties was in fact determined to be one of the key challenges 
for this approach. The evolution process must be sped up and selection criteria 
made more complex to better select for performance. Biology has already solved 
the problem of transport against concentration, so one place to start developing 
directed evolution techniques may be to explore improving membrane 
characteristics, such as balancing selectivity vs. flux. Consider amyloid films with 
a distribution of multiple channel types, each responsible for distinct ion transport. 
The challenges here would include achieving the necessary channel assembly and 
distribution to control membrane permeability. 

One suggested directed evolution approach, referred to by the group as “Bionic 
Spores”, is to introduce unique and possibly non-biological target material to cells 
(e.g. a specific crystalline polymer or inorganic nanoparticles) and let them evolve 
to use it for an improved property such as temperature hardiness. The introduced 
material may then be slowly removed from availability over several generations 
and exchanged for desired feedstock (e.g. the relevant monomer or precursor), with 
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selection pressures intended to favor rudimentary synthetic machinery for the target 
material. This process may be iterated to build up complex machinery that digests 
certain feedstock, breaks it down to the necessary components, and rebuilds the 
target with precision, efficiency and self-sufficiency. 

Another suggested directed evolution approach, referred to by the group as 
“Artificial Darwinian Systems”, requires an initial diverse library of biosynthesis 
capabilities; application of a typically non-required function to stimulate forced 
adaptation; and methods for the selection and propagation of successful candidates. 
The library must link stimulus and response to survival, such as food source 
conversion, but must also be linked to a function that may be traditionally thought 
of as non-biological. Stimulating adaptation may require utilizing additional 
materials to structure the environment to direct evolution towards the target 
solution. For example, if the target is a coating that protects against a given stress 
such as radiation, one method may be to feed a surface of bacteria that are then 
killed and sloughed off via that stress. Selection characteristics must be based on a 
full matrix of biological stimuli and response such as physical, mechanical, 
chemical, optical, electrical, or any combination thereof. The method should also 
have an integrated computational approach to accelerate learning and success. 

ii. The second class includes living systems that generate materials with living 
biology included. These systems may be continually evolving in the field. Target 
materials will need to accommodate demands of living cells such as timescales, 
mass transport, signal transduction, DNA recording and implementation for events, 
and life support requirements such as diffusion of nutrients and energy donors and 
acceptors. Mechanisms regarding materials inheriting properties from the “parent” 
will need to be better understood, as well as how the living components can be 
controlled to be responsive or non-interactive on demand. Long-term stability of 
the living function must also be addressed, such that decedents maintain the desired 
function in the face of exposure to wild-type organisms and operational 
environments. 

iii. The third class includes living materials with dynamically responsive or latent 
activity, such as materials generated that may have living biology added at a later 
time when its function is needed. An example might be a healing (or repair) 
ointment which has organisms in it that are activated at time of use. Catalysis is 
another relevant application that can draw from enzymology. A concept labelled 
the “Infinite Tea Bag” considered cells packaged or engineered to survive extreme 
conditions/storage that can be revived to manufacture proteins or other 
biotechnology product. The packaging of the cells, such as with a membrane, could 
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act as integrated purification of feedstock or product depending on the flow 
conditions during use. 

Challenges 


The workshop included a very diverse group that represented expertise across 
biology as well as chemistry, physics and materials science. Over the course of two 
days of discussion this group identified that a significant challenge to many of the 
biomaterials opportunities is the lack of connection between the research spheres 
of “biology”, which provides much in the way of new design approaches and new 
tools, and “materials science”, which provides not only traditional design 
approaches and tools but also a mature understanding of how to anticipate and work 
to meet application needs. Without direct communication between these groups it 
will be difficult for materials scientists to articulate what a high level application 
would need at the biological level, and biologists don’t necessarily know enough 
about the materials science to know what materials and materials properties are 
useful or novel. One suggestion was for more people to be trained in both 
disciplines to act as intermediaries. In addition, it was suggested that the materials 
science side, and end users, must be educated to be patient so as to allow biological 
sciences to mature to where they may be used to address materials needs, since 
early bio efforts are not likely to solve the problems quickly. “Living materials” 
solutions that have promise to provide complex and highly functional materials 
solutions are anticipated to be even more far future. In the near-term, it was 
recommended that a workshop be implemented to bring together materials 
scientists and biologists with tutorial components to provide a common language. 

Several underlying technical needs were identified as common challenges across 
the biomanufacturing opportunity space. The most prominent are the need for 
fusing multi-scale modeling with bioinformatics and the need for new 
characterization tools that address biological scales, environments and material 
types. Regarding the former, improved meso-scale modeling and other 
computational tools are critical to enhancing our understanding of biosynthetic 
processes as well as narrowing the design space. Bio-informatics need to be 
incorporated to expand design and evaluation capabilities, but given the level of 
complexity in these systems there are current challenges with data analysis that will 
need to be resolved. 

Regarding characterization, there is a lack of imaging capabilities for complex 
heterogeneous materials, systems and interfaces. The ability to determine structure 
and characteristics is limited and resolution is poor for biological, soft and non¬ 
crystalline materials. For example, the suggested direct evolution development of 
semipermeable amyloid films in section 3 above would require angstrom level 
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resolution in order to test functional capability of individual ion channels. 
Interfaces, with relevant interactions at the nanometer scale, are another challenge. 
This is in part because high vacuum and cryogenic techniques are largely not 
compatible with wet systems. Dynamic behavior of biological systems is 
particularly difficult to observe, though neutron scattering is a consideration. There 
is a similar lack of chemical analytical tools to characterize hierarchical structures 
constructed by biology. It is unclear what level of precision is even needed and 
what relevant information can be learned using current techniques. Tools or 
protocols will also need to be developed to screen for materials functionality. 

There is a desire to overcome the current challenges and limits regarding the scaling 
of bioproduction to very large macroscopic structures. There are currently many 
applications on the nanometer to millimeter scale driving research. There is also an 
argument that merely scaling up is not a scientific question and the “market” 
(industry) will take care of it for certain classes of materials that people find most 
interesting. It was determined that this is not entirely correct and there are also 
critical scientific questions within this context. It was also suggested that industry 
taking the lead will depend on the economics, and it is unknown whether the current 
economics of chemical synthesis will hold up or break down as we scale up. The 
value-added proposition for bio-enabled synthesis will be governed in part by the 
complexity of products obtainable and the relative challenge of producing a near¬ 
peer compound through traditional synthetic means. 

There is high potential value to understanding and utilizing various types of 
materials that lie between living and nonliving, for example cells without genomes 
or in vitro systems improved with maintenance factors. There is particular need for 
materials that interface functionally and dynamically between biotic and abiotic 
materials, including controlled assembly and disassembly. To facilitate this and 
other advancements, there is a biotic/abiotic language gap that must be addressed. 

Capabilities and Applications 

This final section addresses end-use capabilities that the opportunities stated here 
could provide. The capabilities may be subdivided rather evenly as those that 
support more advanced research and those that support more advanced end use 
application. 

Regarding capabilities supportive to research, many of the ideas generated have 
already been discussed as part of the sections above as they are largely challenges 
within the sphere of biology. Examples might be a toolkit of biological catalysts 
(such as chaperones, nucleases, and polymerases) to control assembly of non- 
biological materials, and the ability to integrate patterning of cells and 
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(bio)polymers precisely and aperiodic ally. Another example was described in the 
“Bionic Spores”, which utilized the notion of introducing unique material or 
functions to cells, a desirable capability that would further many other applications 
as well. Feeding nanoparticles to cells for them to uptake and incorporate into 
structure or products could yield biologicals that make traditional non-biological 
materials as well as dynamic materials that function using nanoparticles. 

Regarding capabilities relevant to end use application, where controllable 
bioenabled synthesis and assembly provides actual value and successful 
implementation will depend in part on logistical efficiency, environmental 
robustness, and what unique products it can generate not available through 
traditional synthesis and manufacturing efforts. A few ideas are listed below: 

• Understanding the biotic/abiotic interface and controlled assembly, 
disassembly, and reassembly: This capability could enable a very wide 
array of new, tunable and responsive functionality and architectures. At the 
small scale, precisely constructed designer nanoparticles would improve a 
wide range of applications from energy conversion and storage to catalysis 
to navigation (e.g. magnetosomes). Ultimately these technologies could 
improve mobility and weight reduction, such as through devices that are 
more wearable or better integrated with the platform. Coordinated arrays of 
nanoparticles, or larger scale bioenabled structures, could provide 
autonomous sense and response to internal states (stress, fatigue, metabolic, 
pathogenic, etc.) and external states (environmental conditions, incoming 
threats, vibration/sound, etc.). Responses may include changes in 
optical/electromagnetic signature, mechanical, chemical, thermal, 
magnetic, pathogenic, and other properties. These materials, if living, might 
adapt and evolve rapidly to adjust to all external and internal conditions and 
regenerate as needed. There is potential in many of these capabilities to 
minimize the need for human interaction to enable and maintain integrated 
material performance. Smart, robust, evolvable materials could conceivably 
be developed that are adaptive, self-assembling, sensing, scalable, learning 
and communicating. 

• Self-sustaining operation: Bio processes could enhance recycling to 
minimize user signature or environmental impact by enabling more 
effective waste-to-energy efforts and then using the degenerated chemical 
biproducts, and those from other materials scavenged onsite, to create new 
target materials. New biomaterials and their products could also improve 
water purification, a major world challenge in the coming years, including 
new approaches to desalination, heavy metal and radioactive particle 
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removal, and other cleansing actions. Our approach to personal nutrition 
may be radically changed, from sensors that inform us about nutritional 
needs to meeting those needs through tailored or evolved biosynthetic 
pathways. As with sense-and-respond biocircuitry, many of these 
applications demand storage, transmission and stabilization of biotic-abiotic 
interfaces. 

• There is significant opportunity to establish new paradigms in information 
management. Biological materials have the potential to generate, store and 
communicate complex nonlinear data. For example, using DNA-synthesis, 
where single nucleotide accuracy/resolution is not needed, portable archival 
storage of petabytes of data may be realized. There is a ‘raw limit’ of 1 
exabyte/mm 3 (10 9 GB/mm 3 ), with an observed half-life of over 500 years, 
in these materials. Developing readout techniques is a key challenge. A 
possible way forward may be to structure the DNA-based data to enable 
logic strategies for pulling out specific pieces of data such as tags, indices 
and record keeping. The interface may also be developed outside of our 
current bounds of approaching this problem. Remotely brain-controlled, 
including thought controlled, interfaces may yield alternative modes of 
information transfer. Such capability is not inconceivable considering the 
current function of our own neural networks and their generation of and 
interaction with electromagnetic waves, and the wide availability of 
electrically and magnetically susceptible biomaterials. 
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Vehicle-Based Soldier-Autonomy Teaming 


December 6-7, 2016 
Aberdeen Proving Ground MD 

Organizers: Jason Metcalfe, Victor Paul, and Kaleb McDowell (ARL-HRED) 

Introduction 

Even if we do not fight the producers of [these] sophisticated weapons, 
warfare will become more lethal as they export this advanced equipment 
to their surrogates or customers. Crises involving such adversaries will 
unfold rapidly, compressing decision cycles, and heightening the risks of 
miscalculation or escalation. Conflict will place a premium on speed of 
recognition, decision, assembly, and action. Ambiguous actors, intense 
information wars, and cutting-edge technologies will further confuse 
situational understanding and blur the distinctions between war and peace, 
combatant and noncombatant, friend and foe—perhaps even humans and 
machines. (Milley, 2016) 

As illustrated by the above statement by General Milley, the future of armed 
conflict is expected to involve high tempo operations and engagements with 
increasingly sophisticated, near-peer adversaries that will threaten the U.S. military 
ability to achieve overmatch sufficient to maintain operational advantages. 
Conflicts are expected to unfold across distributed, multi-domain spaces that are 
unlike any previously encountered; unique operational challenges will emerge as a 
product of a complex mix of land, air, maritime, space and cyberspace based 
threats. Army teams will need to flexibly respond to challenges in this multi-domain 
battlespace whether in sparse rural or dense mega-city environments across a wide 
variety or terrain. Therefore, we may further anticipate operations to require 
adaptive vehicle-based Soldier-systems that possess a range of capabilities 
extending well beyond those existing in current and traditional military vehicles. 
Future Soldier-system concepts will be persistently influenced by evolving mission 
challenges (e.g., increasingly multi-cultural environments, ever-more advanced 
intelligent systems, dense and growing urban areas), constraints and requirements 
of specific operational approaches (e.g., distributed teams, reduced crew sizes, 
closed-hatch operations, Soldier-autonomy teaming), and the direction of 
technological advances (e.g., machine-augmented and intelligent agent enabled 
task execution, adaptive and individualized Soldier-system design). 

It is recognized that combat vehicles are essential to successful operations and that 
modernizing through the integration of emerging technologies will provide 
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significant advantages to win on future battlefields (ARCIC, 2015). To examine the 
role of future technologies in Army Combat Vehicles, ARLs Center for Adaptive 
Soldier Technologies hosted an Army Science Planning and Strategy Meeting 
(ASPSM) on the topic of “Vehicle-based Soldier-Autonomy Teaming” in 
December, 2016. Over the course of two days, scientists and engineers from Army 
research and development organizations (ARL, TARDEC) met with domain 
experts from industry and academia to examine critical issues facing future Soldier- 
autonomy team operations, envision operational Soldier-autonomy interactions in 
future battlespaces, and identify key capabilities to be enabled through manned 
vehicle-based Soldier systems. The group analyzed the identified capabilities to 
derive concepts of future Soldier-autonomy teams and fundamental research 
required to enable their realization. 

The first day of the meeting began with a brief set of overview and context 
presentations to define the problem space as that of enhancing the interaction of 
Soldiers with advanced systems through concepts of teaming. More precisely, the 
objective was to deeply assess the challenges involved in seamlessly integrating 
advanced (and evolving) vehicle-based autonomy, as well as enabling technologies, 
with Soldiers in such a way as to account for mission dynamics, individual 
variability among teams, and intrinsic state changes within individual Soldiers 
through all stages of mission planning and execution. Following the opening 
session, the participants were assigned to three different focus groups and each were 
given a general operational context to frame their consideration of critical future 
capabilities. The focal scenarios were described as “leader interaction in a rural 
environment”, “presence patrol in a low-density urban environment”, and 
“extraction in a megacity environment”. The focus groups were asked to brainstorm 
potential future capabilities for manned-unmanned teams. For each identified 
capability, the groups discussed the risks and rewards that the capabilities 
addressed, why they believed the capability may succeed, what would prevent the 
success of the capability, and finally, the general timeframe (mid (2025-2035), far 
(2035-2050), and futuristic (beyond 2050)), which its maturation would be 
expected. The first day concluded with all focus groups reconvening and sharing 
the sets of capabilities that were identified. Overnight, the identified capabilities 
were compiled into a single document and disseminated to all meeting participants. 
For the second day of the meeting, the participants were then self-organized into 
four different focus groups, two that were asked to identify fundamental scientific 
research that was required to bring the capabilities to fruition and two that were 
asked to define operationally-situated concepts within which the various 
capabilities would be applied. 
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Capabilities for Future Vehicle-Based Soldier-Autonomy Teaming 


In total, the three focus groups identified 37 future capabilities, many of which 
appeared to have common notional underpinnings. Four of the capabilities were 
identified with descriptive titles, but not as fully specified as the rest. Of the 
identified capabilities, the overwhelming majority (22) were labeled as either mid 
or mid-to-far, 7 were identified as either far or far-to-futuristic, 3 were considered 
futuristic, and the 5 were not given a timeframe (though 4 of those 5 were the 
capabilities that were not defined in detail). In the following we present the 
capabilities as identified by each of the focus groups, organized by timeframe for 
capability maturation. Interestingly, the group that was focused on a leader 
interaction in a rural environment mainly identified capabilities on a mid or mid- 
to-far horizon whereas the other two focus groups identified, in near equal 
proportions, either mid-term or far/futuristic capabilities with few in between (mid- 
to-far). 

Across the discussions, a strong theme emerged that interface technologies, 
comprised of sets of intelligent agents, hereafter referred to as intelligent 
crewstations, will be critical to “team” with Soldiers while executing all phases of 
operations. Advancing technologies for intelligent crewstations will enable mission 
planning, execution, dynamic adjustment, and after-action review for improved 
training. Moreover, this foundational intelligent crew station-Soldier team will 
provide an essential functional unit to enable manned vehicles to team with other 
manned and unmanned assets to maintain survivability and lethality during mobile 
operations across a range of environments. In addition to the intelligent crewstation 
theme, several other strong themes of future capabilities also emerged from the 
meeting: 

• Intelligent Soldier aides for operational training, mission planning, and 
rehearsal 

• Effective real-time maintenance and communication of situational 
awareness across Soldier-autonomy teams resilient to degraded, 
compromised, and rapidly-changing informational conditions 

• Situation-adaptive reasoning, dynamic resource allocation, and task 
management to support collaborative Solider-autonomy decision making 
and performance 

• Context and state-aware autonomy to detect or anticipate operational needs 
and take necessary actions to support the Soldier-squad mission 
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• Heterogeneous unit teaming including direct interactions across mounted, 
dismounted, and unmanned assets that function robustly across the 
information and capability divides between platform-specific mounted and 
dismounted technologies. 

Intelligent Soldier Aides for Operational Training, Mission Planning and 
Rehearsal 

• Re-define or revised Army hierarchy that includes non-human agents 
Identify role of different automations and artificial agents within a relevant 
military command structure 

• Intelligent interface for Army battle command and analysis Provide an 
accurate real-time operating picture by leveraging pre-mission data as well 
as dynamic intelligence gathering for improved situational awareness and 
team facilitation through asset monitoring and management. 

• Virtual-reality based mission interactive briefing Human-interactive virtual 
or augmented-reality based briefing that leverages artificial intelligence, 
with minimal human input, to generate situation-specific scenarios that 
integrate knowledge of cultural norms (e.g. common body language, local 
idioms) and other mission-critical data. Enabled by logging capabilities of 
multimodal, immersive interfaces, may later sub-serve after-action reviews 
and scenario-based trainings 

• Individualized performance assessment Provide Soldiers individualized 
assessment and feedback regarding their current suitability or readiness for 
a given mission based on their state, knowledge, skills, and abilities. 

• Active development of team trust Considered broadly, trust between agents 
includes human-human trust as well as human-nonhuman agents in teams. 
Implicit is the involvement of all team agents in training and learning 
scenarios to build a baseline level of ambient “team trust”, but also to 
consider structuring team dynamics to enable and support active trust 
management schemes when embedded in operational tasks and contexts. 

• Adaptive training agent Specifically, tracks attention in the user as well as 
confusion, and can pause, repeat, and attempt to clarify details that the user 
doesn't understand. Allows condensation of details that the user's 
profile/background suggests they already know and emphasis on known 
areas of opportunity in their skillset. 
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Real-time, Resilient, and Distributed Situational Awareness 


• Immersive, multimodal interface Leverage multiple human sensory 
modalities (e.g. auditory, tactile) to deliver information to team about their 
upcoming mission, such as data regarding culture, environment, mission, 
goal, team capabilities, vehicle state, team members state (physiology, 
psychology). Intelligent, innovative use of space and surfaces in vehicle to 
communicate (in addition to HUD or screen). 

• Multi-modal, agent-based intelligent sensor network Distributed system of 
permanent and/or semi-permanent, agent-borne (Soldier, vehicle), sensors 
for real-time sensing and archival access. Multi-layered to include 
physiological and environmental monitoring as well as higher-level 
inference such as tracking socio-cultural variables of interest. 

• On-vehicle data processing and sensor fusion To provide real-time and on- 
demand information to support real-time situational awareness and aided 
decision making. 

• Adaptive and intelligent inter-vehicle connectivity Use of pre-loaded and 
local transmission between vehicles to enable robustness in corns, sensor 
feeds, and other data transmission to maintain local function that is external 
to and not completely reliant on a larger battlefield or urban network. 

• Natural language-based awareness Comprehensive, adaptive natural 
language component for machine control and for augmenting human 
understanding and decision making. Comprehensive means that, as a sensed 
component and as a command input, it can incorporate gestures, 
expressions, tones as well as how they vary according to the context (socio¬ 
cultural, operational, based on the internal state of the communicator). 

• Context-aware HUD HUD is customized based on user context as 
determined by intelligent agent, mission parameters, current user physio 
status, and direct commands. Commands can include gaze tracking, gaze- 
zoom, brain control, wrist-worn 

Situation-Adaptive Reasoning for Dynamic Resource Allocation and Task 
Management 

• Confidence-based inference of team intention Computational identification 
and classification of individual and group intention for both opposing and 
friendly force with integrated confidence on sensor data, state estimates, and 
the intention predictor. 
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• Human state management Application of proven techniques to influence 
and modify human physiological and cognitive states. 

• Automated contextual and situational recognition Specifically for the 
generation of appropriate smart alerts, queries, imperatives, or to enable a 
system to take autonomous action. 

. Dynamic system adaptation and reconfiguration Enabled by an adaptive 
learning system to respond to nonlinear changes, such as big structural 
changes in the team or its assets while preserving a recognizable command 
structure. Both software (e.g. algorithms, sensor management) and 
hardware (e.g. reconfiguration, replacement) to account for partial or total 
instrument or agent failure or loss. 

• Leverage and adapt existing environment To act on, engage with, and 
modify a given environment through small teams of man and unmanned 
assets that are able to jointly exploit the environmental affordances, both 
implied and by mechanical design, to augment team and compensate for 
capability gaps in the service of current mission needs and overall 
objectives. 

• Iterative human-machine problem solving Enable humans and non-human 
agents to engage in iterative, joint problem solving that will enable faster 
and more robust solutions; leveraging human adaptability and thinking 
outside of rule-bound structures while capitalizing on rapid computational 
reasoning and scenario analysis of intelligent, computationally augmented 
agents. 

• Dynamic load and task management Leverage team and individual state 
inference to detect circumstances of overload or potential overload and 
dynamically re-allocate tasks and roles to compensate. Enable tiered 
strategies to allow provision of the least help that is necessary and sufficient 
to mitigate overload (whether provision of information or complete re¬ 
tasking). 

Squad State and Context-Aware Autonomy for Mounted and Dismounted 
Mission Support 

• Autonomous commandeering Enables leveraging of existing urban 
infrastructure, information, and mobility networks to enable achievement of 
mission and provide physical and information security. 
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• Autonomous dismount support Dismount-support system that 
autonomously can provide medical care and security through active defense 
and shielding. 

• Autonomous asset placement and management System to enable dynamic 
physical support of dismount forces by increasing efficiency in placement 
and accessibility of operational assets. 

• Vehicle-mounted multi-UAY operation center Individual docking ports for 
approximately 5 UAVs enabling optionally-autonomous launch, control, 
and renewal. An intelligent control system should be integrated with the 
vehicle to enable autonomous coordination of the UAV group, including 
monitoring for self-refuel and repair (leveraging a 3D printer and robotic 
assembly kit). Enables as-needed handoff between the vehicle-based 
controller and mounted- or dismounted-Soldiers, leveraging physiology- 
based sensing to enable Soldier state-based semi-automated handoff (e.g. 
during high stress/overload conditions, system can determine high-level 
functions such as “follow”, “return home”, etc). 

• Medical evacuation for patrolling teams. Vehicle deploys flat autonomous 
stretcher for personnel retrieval that contains containment gel and more 
advanced status monitoring. Upon arrival, system wraps wounded 
combatant in a super-cooling stasis gel to limit the risk of battlefield 
bleeding fatality. The structure of the system can then monitor, assess, and 
transport the wounded warfighter, either to the deploying vehicle to a pick¬ 
up point for an autonomous quadrotor device. 

• Proactive, predictive action Using the most salient factors and models of 
likely scenarios, perform reasoning and anticipate behaviors most necessary 
to perform to optimize survivability and mission success. 

True Heterogeneous Teaming, Robust to Information and Capability Divides 

• Human-directable automation Advanced automation that is designed to 
enable human interactions that allow directing and influencing function to 
flexibly achieve desired outcomes. 

• Automated task management system Adaptable system to prioritize and 
share tasks within and among teams of human and machine teammates. 

• Human-automation-integrated team facilitation To integrate all agents in a 
mission using a cohesive framework for clear, concise communication, 
information transmission, and problem-solving that is tailored to the needs 
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of each individual agents state and experience as well as defined military 
command structure. 


• Optimized human-machine decision making Autonomy capable of 
perceiving the physical, psychological, and social state of the crew (to 
include defined operational hierarchy), including individual knowledge, 
preferences, and intent and then, in turn, tailor the level, cadence, and other 
aspects of information transmission to optimize the crew’s situational 
awareness and decision making as well as to tune the response of the 
autonomy to the crew and context. 

• On-board intelligent action in high uncertainty Enables operation without 
network connectivity and/or with greatly degraded information quality and 
team capability. 

• On Board Intelligent Assistant Leveraging on-vehicle processing and multi 
sensor fusion, intelligent (automated) task management and prioritization 
algorithms enable a rich, multi-modal human machine interactive system 
that provides for faster, more effective decision making, offloading of 
cognitive tasks and enables shared situational awareness. 

Fundamental Research to Enable Future Capabilities 

Fundamental research was identified that will be required to enable the realization 
of future Soldier-autonomy team capabilities. Several of the most critical research 
questions cut across major capability themes identified above: 

Human State Characterization: There is a need for better ways of characterizing 
human state (cognitive/affective/social) in ways that are useful to and usable by 
(i.e., that can be reasoned about) by autonomy and other human to facilitate 
mutually-adaptive teams. Underlying this general requirement are research needs 
including: understanding variability across and within humans; understanding the 
limits and the opportunities for new mathematical methods to address real-world 
levels of complexity with potentially unstructured and incompletely characterized 
state space; and understanding the relationships between sensing (characteristics 
and performance properties under operational conditions) and human state 
estimation biases and variability across multiple modalities and scales. 

Enhanced Communication between Humans and Machines: There is a need to 
develop better approaches to exploit human multisensory capabilities and share 
concepts for effective communications between humans and machines. Underlying 
this requirement are research needs including: cross-modal approaches to enabling 
real-time human comprehension under constraints of bandwidth, information 
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quality, and task; predicting the effects of individual Soldier variability in sensory 
perception and utilization, as well as, the effects of variability arising from 
operational/technological factors (such as clothing, headsets, crewstations) on the 
ability to comprehend information; and developing approaches to share concepts 
across the information and capability divides of platform specific manned, 
unmanned, and dismount technologies. 

Linking Individuals to Team Behavior: There is a need to understand the mutual 
influence and adaptations between individual and team behavior. How does an 
individual impact team behavior and how does team behavior impact the 
individual? Teaming is not about boxes and functionality, it is about 
interdependence among the boxes. Accounting for interdependence requires 
observability, predictability, and directability. Perception and adaptability provide 
feedback among these three elements and define research requirements including: 
develop the interface between observability and predictability; understand how a 
transaction between humans and intelligent systems occurs; develop tools that help 
identify parameters critical in collaborative behavior; develop tools to understand 
the psycho-physiology of intelligent systems and hybrid teams; and develop 
meaningful measures to assess team behavior. 

Robust Human-Autonomy Integration: There is a need to understand approaches to 
match human capabilities with the capabilities of rapidly advancing autonomies. 
Underlying this general requirement are research needs including: identifying the 
critical attributes of agents that would successfully integrate in the crew-automation 
environment; understanding human-autonomy integration in terms of recruitment, 
assembly of teams, understanding how to establish training protocols and potential 
removal from service or prevention of inclusion; developing novel approaches to 
integrating humans and autonomy focusing on non-traditional tasking and decision 
making processes; principles to develop an adaptable user interface to facilitate 
tuning to individual variability (biases, proclivities, capabilities); principles and 
ethics underlying the generation, evaluation, and refinement of adequate mission 
plans for distributed human-autonomy teams. There is a need to define the kind of 
assumptions can and should be made considering long-term societal shifts in the 
relationship between humans and automations, taking into account anticipated 
perfusion of sensor, computational, and control technologies throughout daily life. 

Human-Autonomy Teaming Analysis: There is a need develop the requirements for 
tools to help design and build human-machine systems? We have tools to analyze 
automation and we have tools to analyze human performance; however, analyzing 
teams of humans and intelligent systems is not a linear combination of these two 
and the current methods and approaches to analyzing heterogeneous teams of 
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humans and autonomous agents remains deficient. Underlying this requirement are 
research needs including: developing system engineering practice and models to: 
better integrate the human, account for dynamics of work, team, individuals, 
environment, better account for the concept of time, present alternatives, and 
critically, account for intelligent systems working collaboratively with humans. 
Further sufficient optimality criteria need to be defined, implemented with 
appropriate metrics for human-autonomy team success. 
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Materials for Sustainable and Mission Flexible Intelligent Systems 


December 8-9, 2016 
Aberdeen Proving Ground MD 

Organizers: Dr. Shawn M. Walsh (ARL/WMRD), Dr. William L. Benard 
(ARL/SEDD), and Dr. Ivan C. Lee (ARL/SEDD) 

Speakers: Preference was given to maximizing input of all attendees, and 
minimizing “seminar” style presentations. As such, only five speakers were invited 
to present, and they were chosen primarily as active representatives of the three key 
themes that defined the bounds and scope of the ASPSM meeting: 

• Robust Self-Sustainment in Austere Environments - Enablers for 
Comprehensive Power and Energy Efficiencies - Professor Dionisios 
Vlachos, University of Delaware 

• Actuation and Maneuver as a Fundamental Materials Challenge - Professor 
Rob Shepherd, Cornell University 

• New Synthesis and Process Enablers for Multifunctional Power, Mass, and 
Volume Efficiencies - Professor Michael McAlpine, University of 
Minnesota 

• ARL Intelligent Systems Vision for 2030+, Dr. Brett Piekarski, ARL 

• Soldier Users Community Perspective on the Potential and Current Limits 
of Unmanned Systems, Mr. Keith Singleton, Chief, Unmanned Systems 
Team, Maneuver Battle Lab, Fort Benning, Georgia. 

Background: 

On December 8 th and 9 th , 2016, ARL hosted an Army Science Planning and 
Strategy Meeting (ASPSM) on “Materials for Sustainable and Mission Flexible 
Intelligent Systems.” The meeting was based in part on the hypothesis that 
integrated innovation in materials, synthesis, and energy have a fundamental role 
to play in disruptively advancing desirable and necessary intelligent platform 
capabilities and behaviors over a range of moderate to extreme operating 
conditions. However, this hypothesis also maintains that advances in “classical” 
materials, synthesis, and energy may not be sufficiently capable of expanding 
beyond their own inherent limitations without new and meaningful connectivity to 
other, perhaps disparate, physics, chemistry, biology, and computation phenomena. 
Every ASPSM invitee is a noted pioneer in fields and activities related directly and 
indirectly to the ASPSM meeting; indeed, it was the significance of their 
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professional accomplishments to date that made them especially critical to helping 
define where the current gaps - and opportunities - lay in this area. The goal of the 
ASPSM meeting was to create a conversation that would allow for materials, 
synthesis, and energy to be discussed explicitly and simultaneously in the unique 
context of intelligent systems. Mass, volume, and energy efficient bounds defined 
the fertile space for identifying the knowledge gaps and transdiscipline work that 
needs to be done so that offsets in intelligent platform performance could be assured 
deep into the Army’s future. 



Computation Sciences 


Human Sciences 


Materials Sciences 


Biology 



New, energy efficient 
& intelligent agent 
maneuver 


New, energy efficient 
& intelligent agent 
actuation 


■ 


Effective 
"human/agent 
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Figure 1. Army Science Planning and Strategy Meeting Goal: Attempt to promote 
transdisciplinary thinking as it relates to intelligent systems and robotics 

Objective & Scope: 

To date, research in robotics and autonomous systems has been driven by the “hard 
problems” associated with advances in greater (and true) autonomy, artificial 
intelligence and device control, sensor development and data fusion, object 
recognition and navigation, multi-agent coordination, and human-machine 
interfacing, training, and teaming, among others. Concomitant with these thrusts 
has been pioneering research conducted from a materials, synthesis, and energy 
perspective, and highly novel intelligent system behaviors and performance 
characteristics have been demonstrated with varying levels of success and 
technological maturity. However, although some energy efficient “behaviors” have 
been identified (including those imitated synthetically or mechanically from 
nature), many of these material concepts have been developed with little or no 
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bounds on mass, volume, and energy efficiency - or if they have, the bounds have 
been rarely comprehensive. Some of these bounds are shown in Figure 2. This 
ASPSM meeting challenged participants to think of the “penalties” associated with 
achieving desired intelligent system and platform capabilities upfront , so that 
fundamental research opportunities and knowledge gaps could be identified to 
break through limitations on current platform performance (e.g., mission 
endurance, durability, maneuver and actuation in extreme terrain and airspace, etc.) 



Challenges: 

> Operate in austere environments (e.g., limited energy 
sources/options) 

> Operate in logistically limited/denied environments 

> Operate in difficult mobility/maneuver environments 
(e.g., irregular terrain, extreme weather, subterranean 
and dense urban environments) 

> Operate in environments where excessive noise can 
compromise Soldier and the mission 

> Minimize weight burden to human Soldier 

> Rapidly adapt functionality to enable mission-needed 
capability 


Figure 2. Identifying the Bounds on Performance of Intelligent Systems to Inform 
Fundamental Research and Knowledge Gaps 

Equally important however is identifying how fundamental advances in materials, 
synthesis, and energy could enable entirely new modes of intelligent system 
capability, behaviors, and responses on different length and time scales. For 
example, prior ASPSM meetings cited “resiliency” as critical in both individual 
and multi-agent systems (e.g., swarms, flocks, fleets). New and comprehensive 
approaches to materials and energy could span the spectrum - from enabling 
radically “attritable” systems to developing new and complex behaviors that give 
disruptive strategic and tactical advantages in Soldier/robot training and teaming. 
These advances not only are strategic from a military perspective; they also have 
the potential to dramatically expand competitive advantages in the commercial 
sector use of robotics and intelligent systems. As such, new insights and invention 
in this area could be equally vital to both U.S. economic growth and the ability to 
domestically and rapidly manufacture DoD-critical materiel - including sustainable 
and mission flexible intelligent systems. 

It is worth defining “intelligent system” for the purposes of this ASPSM meeting 
given the broad lexicon and available terms commonly used (e.g., robots, 
autonomous systems, semi-autonomous systems, unmanned platforms, optionally 
manned platforms, drones, intelligent agents and systems, etc.) While the ASPSM 
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discussions, insights, outcomes, and recommendations could be applicable to a 
richer cross-section of robotics (e.g., discrete platforms, human-machine interfaces, 
powered exoskeletons, and industrial robotics) the scope was primarily bounded by 
the desire to address knowledge gaps and critical limitations on air and ground 
intelligent systems. As such, a working definition for “intelligent systems” for this 
meeting was: devices for the remote projection of a desired capability with levels 
of decision-making ranging from total human control to total device autonomy. 

ASPSM Day 1 Meeting Strategy 

The first day of the meeting began with immediate and unstructured engagement of 
all invited participants. All participants - to include those from academia, industry, 
other government laboratories, and the Army Research Laboratory and Army 
Research Office - were asked to describe their “hardest problem” - that is, what 
they would like to do, but cannot at this time. The only constraint was that the 
participants attempt to link their hardest problems to one or more of the three key 
workshop themes: 

• Robust Self-Sustainment in Austere Environments - Enablers for 
Comprehensive Power and Energy Efficiencies 

• Actuation and Maneuver as a Fundamental Materials Challenge 

• New Synthesis and Process Enablers for Multifunctional Power, Mass, and 
Volume Efficiencies 

This round table discussion was then followed by aforementioned invited 
presentations. Since we had a diverse set of invited participants who may not have 
depth in all three key theme areas, these presentations helped provide context. The 
audience was encouraged to ask questions of all speakers to provide first-hand 
access to the academic and the two extreme ends of the Army “intelligent system” 
spectrum (i.e., R&D and User Community). 

A very candid sampling of the “hard problems” and shared insights transcribed 
from notes and output recorded during the informal engagement with ASPSM 
attendees over the two day meeting include: 

“I think it is important for us to expand our knowledge of the dynamics of materials 
properties, especially mechanical, in the context of energy storage, energy release 
at high rates, and shape and structure transformations. ” 

“Power and propulsion are key. Without them, robots are going nowhere fast, and 
[will be of] limited strategic value. Strap a jet engine to a brick - and you’ve got 
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some very real capability. How do [we as a research community] advance 
intelligent system propulsion and power? 

“Soldiers often have to carry everything they need like armor, batteries, 
ammunition, water, and now, some [man-portable] robots into [remote] locations. 
Remember that. That is why reducing weight [of these items] is so important for 
Soldiers and key to [Army User Community] acceptance of new technologies” 

“There is a trend [in the research funding agencies] to mandate modeling, 
roadmaps, “tool development”, etc. in proposcds for research in areas like this. 
Those may only lead to lots of information on inferior concepts or materials etc. 
More room is needed for exploration. Models and tools will flow better and be more 
useful after discovery...breathing room for new insights and discovery is needed. ” 

“[We can already] dramatically shift elastic [monolithic] material moduli now. But 
so what? What useful bulk properties or useful [robotic[ response has that recdly 
given us? Maybe some useful [applications[ in soft robotics. We need to think well 
beyond pure material property development, and maybe do it in [context[ of 
unprecedented morphologies and multiscale materials mechanisms to give us 
breakthrough response times and levels of actuation force that we just don’t have 
right now. ” 

“My ‘hardest problem’ is a cultured one [in robotics research]. How do we work 
basic science and engineering together, with people right across the hall [at my 
university]?” 

“We have tools we didn ’t have 10 or 20 years ago. Additive manufacturing may be 
at risk of being oversold, but recdly we are at the dawn of this type of capability. 
We can and will continue to achieve absurd complexity that can enable [entirely 
new] approaches to [intelligent system] design, materials and power. [Synthesis] 
and manufacturing could be the nexus of many of the advances we are talking about 
[at this meeting]. 

“I could not help notice that the line between “smart materials” and “robotic 
materials” becomes very, very thin in some areas. [But] it does seem like there are 
clear distinctions and linkages, especially in reflecting the loeed and globed 
intelligent system needs and required capabilities. ” 

“[Simply developing tools and research for] mimicking nature is a waste of time 
for robotics. Nature isn’t edways that efficient at everything, but it is often good 
over a range [of behaviors and phenomena]. [The key is] being inspired by nature 
to identify the [complex] approaches used to achieve efficient material and energy 
use and adaptivity in the environment where [living organisms] live and flourish. ” 
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“One take away from the meeting that struck me was that, since the field is evolving 
and highly interdisciplinary, it seems to be lacking basic understanding of physical 
constraints. I said as much during the wrap up sessions. I was shocked that some 
argued that solar energy could supply the needs of advanced robotics, and they 

dismissed higher energy density mechanisms. It was very telling . But I found that 

some in the audience were unable to understand concepts like energy and power 
density. This is a huge limitation to advancing the field. ” 

“We need to distinguish what we mean by “computation” and “analytics. ” Both 
are important to robotics as [intelligent systems] evolve. Also bringing 
computation and control into the materials to [enable them to] shift properties or 
morphologies on different length scales - that’s an exciting place to be if we want 
new [approaches] to materials and energy use [in intelligent systems]. ” 

“[There is] a joke from the comedian Mitch Hedberg: “An escalator can never 
break: it can only become stairs. You should never see an Escalator Temporarily 
Out Of Order sign, just Escalator Temporarily Stairs. Sorry for the convenience. ” 
[He] definitely [has] the right idea. If you can have a device that has one very useful 
behavior when it’s powered, and another useful behavior when it’s not powered, 
that could be powerful in the design of systems in which power is limited. You could 
think about intentionally staying in the “no power” mode even when you have 
power, to conserve for when you really need it. ” 

Day 1 Breakout Session Strategy 

The afternoon, and the bulk of Day 1, was devoted to breakout sessions that were 
focused on the three key ASPSM theme areas. 

Orange Team: Robust Self-Sustainment in Austere Environments - Enablers for 
Comprehensive Power and Energy Efficiencies 

Yellow Team: Actuation and Maneuver as a Fundamental Materials Challenge 

Green Team: New Synthesis and Process Enablers for Multifunctional Power, 
Mass, and Volume Efficiencies 

“5+2” Breakout Session Brainstorm Questions: All participants were given a set 
of 5 predetermined questions to use as basis for brainstorming. They were also 
asked to develop 2 “new” additional questions. Finally, they were encouraged to 
replace any of the 7 (“5+2”) questions in favor of ones they felt were more 
provocative and effective. The only constraint imposed was that each session 
remain faithful to the theme of the breakout session to which they were assigned, 
but outcomes could definitely link to other theme areas outside their breakout 
session. ASPSM organizers served as facilitators for each of the breakout sessions. 

Approved for public release; distribution is unlimited. 

78 




1. What are the fundamental properties of robust actuation materials (robust 
in terms of operational environment and performance)? 

2. What are the key elements of holistic energy efficient actuation (mechanism 
scalability, match to desired effect, self-sustainment)? 

3. What are the physical performance bounds? 

. Mass 

• Energy Density 

• Idling Penalties 

• Operational Environment 

• Maneuver 

4. How do we balance complexity of synthesis/assembly with performance? 

5. How do we balance mission specific performance with robustness? 

Day 2 Session Strategy 

After reconvening on Day 2, the ideas developed from Day 1 breakout sessions 
were discussed in more detail as an entire group. The goal was to begin to identify 
common themes, group them where it made sense, and provide more cohesive and 
succinct research recommendations. The caveat was to remain faithful to the three 
themes that defined the unique scope of this ASPSM meeting, and to use the Day 
1 breakout session outcomes as a basis for expansion and refinement of knowledge 
gaps descriptions. Hence, although the three workshop themes are implicit in all of 
the outcomes below, it made more sense to provide new titles that better describe 
the cumulative insights and areas identified by the breakout session groups and 
individuals alike. 

• Upsetting the Autonomous Applecart: New Modes of Mobility, Actuation, 
and Manipulation 

Roboticists are already actively exploring and addressing the problems of motion 
and actuation from a physics standpoint, as well as making the linkages to other 
elements (e.g., design, power, computation, materials, etc.) to develop functional 
systems. For example, the term robophysics 1 is well suited to describe the overall 
framework for many of the areas discussed in the ASPSM meeting - including the 
“intersection of robotics, soft matter, and dynamical systems 1 .” Indeed, the Army 
has funded and supported such efforts 2 across a variety of internal and external 
research efforts, and collaborations with academia and industry. Many of the 
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ASPSM participants have pioneered not only terms like “robophysics” and “robot 
blood 3 ” but also have been part of efforts to more comprehensively inform 
intelligent systems capability through interdisciplinary science and technology 
integration. However, the goal of this meeting was to identify the gaps and 
challenges that preclude further advances in this space, as well as the opportunities 
that may now be possible. Some key insights from breakout sessions include: 

o Currently, there is heavy reliance on very conventional, inefficient, and 
power hungry electromechanical systems for enabling locomotion and 
actuation. Many of the innovations are achieving complex behaviors 
(snake-like mobility through confined spaces, adaptivity to solid and 
granular surfaces, soft, conformal gripping of objects, adaptive aerofoils for 
low power loitering). Much of the integration is still conducted at the macro 
level - and not addressed as a more fundamental material or true 
multifunctional materials problem 

o Achieving reliable and repeatable levels of desirable motion, mobility, and 
manipulation rapidly becomes challenging as the length scales become finer 
and the devices become smaller. Yet nature seems to operate just fine (e.g., 
insects) on smaller length scales, as well as in the face wind, rain, and other 
adverse and unpredictable environmental influences. 

o With notable exceptions many roboticists (and robophysicists) designing 
and synthesizing robots are using materials that are fairly conventional, if 
not “commercial off the shelf’ or antique. Conversely, and more 
importantly, they [roboticists] are not always fully aware of what materials 
and energy expertise can bring. The idea of “kits” was suggested - to allow 
materials scientists to supply or tune materials unique to robotics 
applications. Earlier transdisciplinary research connectivity is equally 
important, so that robotic modes and materials could be conceived in 
tandem. There is significant value in bridging the Robotics Community with 
the Materials and Energy communities. Historically, the Robotics 
community has focused on mechanisms and algorithms to demonstrate 
concepts. Advertising the full capability of new materials and energy 
systems both in terms of properties, as well as access and design rules for 
synthesis, could lead to new breakthroughs. Ideally a library of materials 
and mechanisms for actuation would be developed - to enable a dialog 
between materials scientists and robotics community to find better materials 
and multifunctional responses. 

o Absent in many of the approaches towards locomotion and actuation in 
intelligent systems and robotic devices are alternative approaches to energy 
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and power - including a priori assessment of the energy penalties for 
achieving performance. There is not a cohesive or fundamental approach 
for enabling much higher energy densities in the context of robotics. There 
is a gap in (and need for) more direct and efficient energy conversion 
mechanisms to enable actuation, locomotion and manipulation. 
Specifically, there is a gap in developing materials that could support new 
modes of intelligent, energy-dense and efficient actuation (e.g., materials to 
contain caustic chemical reactions for energy conversion, micro¬ 
combustion, materials for flexible, multifunctional fuel cells, multisource 
energy harvesting, etc.) 

o There is a gap in developing distributed control within materials and 
structures, and a need for concurrent and holistic design of structures and 
materials. 

o Develop combinations of both materials and energy fields to enable 
actuation and manipulation. The use of fields to change states of matter is 
not new. But the relatively weak, sloppy, or power intensive nature of 
“conventional” approaches (e.g., electrorheological fluids, 
magnetorheological fluids) often make them impractical and confined to a 
lab bench novelty. If materials could be designed in such a way as to allow 
for rapid and reversible changes in properties (e.g., load bearing “switches” 
made in the material itself that could be readily realigned from one state to 
another by strategically applied fields or energy) it would be an enabling 
benefit to actuation and dexterous manipulation. The goal would be a more 
adaptive, energizable version of a “universal gripper” by thinking deeper 
about the morphologies of the materials, and how the particles or 
substructures can align or disperse with localized and controlled application 
of fields (pressure, vacuum, electric, magnetic, etc.) 

o Vastly improve the ability of organic and inorganic materials to store and 
release mechanical energy; new tools and concepts to manipulate the 
complex moduli of materials. 

o Engineer materials so they can be readily broken down and reassembled 
with minimal energy required to move from one state to another -and then 
back to the original state (highly reversible, with little or no loss). 

o Noise and excessive heat generated by robotics systems are often 
afterthoughts and may be acceptable for, say, manufacturing robots. But for 
the Army, such operating artifacts of intelligent systems could compromise 
a Soldier or the mission because these artifacts are easily detected. Early 
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and fundamental consideration of noise and heat generation as a materials 
and energy problem in the context of intelligent system mobility and 
actuation is warranted, and as evidenced in the recent Marine’s dismissal of 
a “Big Dog” variants due to noise, it is a current gap 4 . 

o Robot performance is fundamentally limited by the availability of 
conventional bulk materials to convert, store, and release energy 5 . For 
example, robot performance is limited by the power density of actuator 
materials and locomotion stability can depend on the passive dynamic 
properties of structures such as legs and feet. Current bulk materials are not 
matched to the performance needs of robots; for example piezoceramic 
actuators have very low strain while dielectric elastomers actuators have 
high damping due to fundamental material properties. By constructing 
metamaterials at the nano, micro, or milliscale, novel mechanical and 
physical properties could be incorporated into individual "domains". By 
organizing 2D and 3D arrangements of these domains, novel properties 
could in principle be preserved in bulk metamaterials, and appropriately 
scale. Example metamaterials for robotics could include transducers which 
convert fields to mechanical energy, thermal to mechanical energy, or 
convert chemical to mechanical energy. Other metamaterials could have 
mechanical properties such as stiffness or damping which are easily tunable 
on demand, either at fabrication time, in the field, or dynamically while in 
operation. Research is needed into understanding the physics, design, and 
fabrication of these proposed metamaterials. 

o While a strong and sensible case can be made for moving away from 
discrete and fairly conventional devices and mechanisms to purely material- 
based actuation, there are knowledge gaps and current bounds on 
performance that limit this desired capability. These include the ability of 
so-called “smart materials” to generate any significant amounts of force to 
enable an equally significant amount of work. Also, the responsiveness of 
many materials to imposed stimulus is not robust or adequate. The materials 
might be very responsive upon stimulation to shift from one state to another, 
but they are often unacceptably slow in reverting back to their prior state. 
This is a serious limitation, as mentioned earlier from a materials 
perspective. Truly reversible and responsive materials and energy coupling 
is needed. Responsive materials coupled with responsive structures enabled 
by low power to generate globally responsive macrostructures; and 
reconfiguring the dynamics and kinematic properties of the materials, are 
current gaps. 
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o How do we inform the hierarchy of actuators, sensors, energy, and 
structures to “turn on” and “turn off’ at their optimal length and/or time 
scales? 

o Materials that retain their extreme “stretchability” and elasticity over 
extreme temperature ranges are highly desirable and elusive currently (e.g., 
10% strain, >400°C) 

o High Efficiency Chemo-Mechanical Actuation: Leverage extraordinary 
energy density of chemical fuels and materials: develop tools to process 
efficiently into motion; develop architectures for scalable robust actuation; 
“robot blood” distribution and caching of energy 

. Beyond Additive Manufacturing: Multiscale Synthesis, Processing, and 
Fabrication of Material and Energy Management Infrastructure Unique 
to Intelligent System Needs 

o Gaps exist in achieving multimaterial integration on different length scales, 
and this multimaterial integration should be expanded to include materials 
used as energy sources, computation, and local and global sensing. Current 
synthesis, processing, fabrication, and manufacturing science offers hope, 
but these areas are often an afterthought in the current robotics community. 
A truly holistic approach, across design tools, materials, energy, and robotic 
science is needed. We can fabricate morphologies and structures and 
components on different length scales that we have never been able to 
before; so why aren’t we? 

o A serious gap exists in the quality, complexity, and true multifunctional of 
materials and components that can be additively manufactured by current 
synthesis and fabrication methods. A novel alternative would be to explore 
much more aggressive innovation in the type of materials and 
subscale/microscale components that are fed into these additive processes. 
These materials could be “engineered packets” that contain structural, 
computational, sensing, and power elements. They could be metamaterials, 
multifunctional materials, energy dense materials, and even robot blood. 
These “robopackets” could be fundamentally designed in such a way as to 
allow for rapidly configuring and reconfiguring them into new and desired 
components for the intelligent system or robot. It would also be necessary 
for these packets to be readily disassembled. Rapid disassembly of materials 
is a neglected knowledge gap in processing. Disassembly of finite robotic 
materials would enable them to be repurposed in new and needed structures 
required by the robot or intelligent system. There is, currently, very heavy 
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emphasis on “additive manufacturing” but it would be desirable to have a 
fabrication process that is capable of both additive and subtractive influence 
on materials - including the robopackets. Thus, simultaneous innovation in 
the development of both the robotpacket concept and the additive and 
subtractive process concept could provide entirely new means to efficiently 
achieve hitherto unavailable complexity in bulk material and component 
response for the intelligent system. 

o Many current methods for synthesis (from chemical scale to additive 
manufacturing) do not get below defect densities sufficiently or reliably. 
Eliminating defects is a limit in general on processing, but also key to 
ensuring more reliable and complex material response for intelligent 
systems. 

o Microrobotics have a significant role to play in enabling new modes of 
macro-system capability and manufacturing and processing of materials, 
and would benefit from advances in more efficient materials, power, and 
control for actuation on very small length scales. Microrobotics also have 
the ability to reconfigure fixed amount of materials and energy resources to 
enable the needs of the larger, global, “macro” robot or intelligent system. 
And microrobots could be dispatched to bring back material and energy 
resources from the operating environment (drone/worker bee analog) 

o Though early, it is possible to “print” and “reprint” materials and custom 
infrastructure directly on the human body. What new benefits does that open 
up for improving human-machine interfacing and teaming? 

. Eat Out or Dine In? Expanding the Menu Options for Efficiently Feeding 

& Sustaining Intelligent Systems in Austere Operational Environments 

o The lack of a comprehensive approach to energy and power storage, 
generation, acquisition, and strategic management and use in the context of 
intelligent systems was consistently identified by all breakout groups as a 
critical knowledge gap. Energy is precious and critical for intelligent 
systems to exhibit decisive and swift execution of tasks and maneuver. Gaps 
exists in the development and integration of (1) multifunctional materials 
for better energy storage, (2) material and morphologies for energy and 
power “infrastructure” (3) intelligent local and global energy management 
in the intelligent system; (4) expeditionary and environmental energy 
acquisition and replenishment; (5) situational awareness, tools, and data 
integration for apriori mapping and assessing available energy resources in 
the mission operating environments (e.g., natural materials or man-made 
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(power lines, fuel depots, waste, scrap, etc.)). A first attempt to 
schematically present some of these ideas and relationships is shown in 
Figure 3. 



(a) Intelligent Platform as "closed" self-sufficient system: cannot acquire 
or modify any material or energy from local operating environment 
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Expanded In-situ Task 
Adaptability & Endurance 


Fixed and renewable/augmentable 
energy resources 


(b) Intelligent Platform as "open" self-sufficient system: ability to favorably interface with 
local environmentto get its needs met and adapt functional elements to accomplish task(s) 


Figure 3. Aggressively, fundamentally, and comprehensively rethinking energy and power in 
all aspects of intelligent system design, operation, and mission environments 

o The concept of “self-feeding robots” is attractive in the context of enabling 
intelligent systems for self-sustainment in austere operational 
environments. This notion of self-feeding robots is not new. DARPA’s 
“EATR” program 6 for example, sponsored research in this area but the 
connection to materials and energy was only partially addressed, without 
fully identifying the gaps - and opportunities of this approach. Newer 
approaches, such as “synthetic metabolism” could work in meter-scale, and 
tens of cm- scale, but maybe not so efficiently in sub-mm scale. Technology 
gaps exist in enabling the “fuel” (high energy dense chemical compound) 
being generated from biomass by foraging. Similarly, the use of 
chemical/catalytic effects and electromagnetic effects for propulsion for 
sub-mm scale intelligent systems is very likely to be challenging for 
meeting the energy requirements for meter-scale device. 
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o Robots and intelligent systems are currently made from materials. What if 
they were made from a combination of materials and energy? What if some 
of the structural materials of robots could also be used as consumable 
energy sources - i.e., a “self-cannibalizing” robot fuselage or component? 
The benefits could be significant and truly unique to robotics in that the 
robot “platform” might continuously change its shape and structure over the 
duration of its mission subject to the constraint that it preserves its ability to 
perform its required tasks when and where needed. Such a concept would 
create interesting knowledge gaps to enable intelligent design and 
intelligent degradation of such robotic structures. 

o In many current intelligent systems and platforms, the losses associated 
with energy conversion into usable work by the robot or intelligent system 
becomes significant if not limiting as the scale of these devices becomes 
smaller and more compact. Need to breakthrough these limitations. 

o A “gradient” of chemistries could efficiently provide different responses 
(with different time constants). Similar, it is desirable to enable structuring 
of many spatially and temporally discrete phase transitions (e.g., materials 
jamming, enthalpy of fusion) for effective and continuous bulk response 
(thermodynamics and kinetics of robots) 

o Explore new modes of “robotic motility” in the context of spontaneously 
linking mobility and actuation with energy consumption more directly. This 
would have broad applicability, but especially in making vastly smaller 
intelligent systems and robotic devices more energy efficient. Biology uses 
motility with great success. What are the beneficial synthetic analogs here? 

o There are at least 3 length scales of intelligent systems: (a) sub-mm, (b) cm- 
scale (c) meter-scale. We often assume we can solve the energy technology 
problem by scaling. That is not quite true (currently); we cannot optimally 
address these three length scale categories with the same energy technology 
in many instances. 

o Many renewable energy sources cannot alone provide the type of energy 
density or availability needed to power a range of intelligent systems. One 
approach would be to broaden the renewable energy palette - create a 
renewable energy portfolio and capacity for the intelligent system to harvest 
energy in multiple modes and from multiple sources. While advancing 
efficient solar energy collection materials and mechanisms should continue 
to be included, there are entirely different modes of harvesting energy that 
need to be explored. This diverse energy harvesting should be constrained 
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by the materials and assets reliably available to the intelligent system to 
ensure efficiencies (e.g., the notion of multifunctional energy harvesting 
materials and mechanisms is a technology gap that could provide strategic 
offset gains in performance and endurance of the robots and intelligent 
systems). New modes of rapid biomass conversion for robots (e.g., 
catalysts, conversion, and process infrastructure) is but one example of 
another harvestable energy resource. The ability to engineer distributed 
chemistries to enable distributed power at point of use is consistent with 
these recommendations. Harvesting thermal gradients and other 
thermodynamic potentials occurring naturally or artificially (i.e., human- 
made) in the operating environment is also another mode to consider more 
deeply. 

o Biology paradigms could be very insightful - the robot as an organism 
hosted by energy resources available in the environment would define new 
relationships and drivers for self-sustainment. 

o Highly flexible organic materials that can withstand extreme and rapid 
temperature and pressure variations and maintain structural integrity could 
support new approaches for micro-combustion for energy efficient 
actuation and mobility. For example, microcombustion in elastic materials 
could enable storage of the kinetic energy as potential energy (in the elastic 
membrane or material) and allow for slower and more controlled release 
and use of this energy by the robot over time. It would provide efficient, 
direct conversion from chemical to mechanical energy and work. And the 
“stretchable” membrane material could contain other functionalities such as 
sensing, plumbing (e.g., microvalves), piezoelectric-like materials, 
computational and control elements, etc. 

o Although the scope of this ASPSM meeting was primarily on discrete 
intelligent systems (e.g., an autonomous ground or air platform) it was noted 
that improving material interfaces between humans and machines could 
significantly improve energy efficiency of robotics used in human 
augmentation (e.g., powered exoskeletons). There is a lot of loss due to poor 
adaptability of these material interfaces, and these add to the overall energy 
and power burden of operation (and limit duration) 7 . 

. Beyond Platforms: Robots and Intelligent Systems as Continuously 

Evolving, Adaptive, and Self-Sustaining “Ecosystems” 

Of all the tools conceived by humans, robots and computers are among the most 

unique. They are tools that have the capacity to create their own tools. And as 
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they become increasingly if not exponentially capable by the confluence of an 
ever-broadening wave of technological advances, they will have the capacity to 
design tools faster and better than humans. This includes tools and “design 
solutions” for materials, components, and systems that may defy human 
intuition but optimally conform to the laws of physics. In addition, while 
natural, biological evolution has demonstrated breathtaking ability to aid living 
creatures to adapt to changing environments and needs, natural evolution is 
relatively slow. In theory, robots - with the aid of artificial intelligence - could 
evolve nearly instantly. Thus, a knowledge gap exists in moving beyond the 
current “fixed, as-designed” intelligent system platform to an intelligent 
platform that has the ability to continuously redesign and reassemble itself 
depending on the demands of its tasks and constraints of its environment. 
Instead of maneuvering with wheels or tracks or rotors (or like a snake, crab, 
dog, or a bird) the intelligent system could assess the optimal mode of maneuver 
- and then reconfigure its constituent material, structural, energy, sensing, and 
computational components as needed. If wheels made more sense than legs or 
tracks, it could create those elements from materials within itself, or from the 
environment, or both. In that sense, the intelligent system becomes less of a 
platform and more of a dynamic “intelligent ecosystem” with the ability to 
intelligently adapt throughout its mission and operational life cycle. Enablers 
for such a concept could include: 

o Rapidly reprogrammable material properties using externally applied 
energetic fields 

o Robot Circulatory systems - The notion of circulatory systems is not new 
in the context of robotics. However, there are new, heretofore unavailable 
concepts and capabilities that would warrant exploration of ideas like robot 
circulatory systems and “robot blood.” Robot blood implies the ability to 
carry energy and materials throughout the intelligent ecosystem. Advances 
in additive and other manufacturing processes could be one of many paths 
to traversing these capability gaps. 

o Modularity vs. heterogeneity. There are benefits to seeking breakpoints in 
materials and design that support reuse of components and materials, or 
even reconfiguration. If the mechanism is simple enough, it could serve as 
the basis for self-healing, or alternatively could consume self as mission 
progresses, shedding functionality no longer necessary. However, 
modularity can separate function on scales that preclude degrees of multi¬ 
functionality - risk under emphasis of system view. These concepts would 
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give the intelligent ecosystem the flexibility it needs to adapt and respond 
as needed. 


o Create new tools for linking intelligent ecosystem terramechanics and 
aeromechanics to the operational environment to deliver efficient modes of 
ground and air maneuver. 

o Point of use fabrication as part of the intelligent system. A logical step 
beyond point of use manufacturing of components for robots by a forward 
deployed manufacturing capability would be to incorporate the ability to 
fabricate within the intelligent ecosystem itself. The robot contains its own 
mobile “factory,” and like all other elements of the intelligent ecosystem, 
that factory (and its tools and processes) are also reconfigurable. 

o If we can bring GPS and other forms of advanced information, sensing and 
computation into material particles, this opens up new ways to organize 
and reorganize materials in the operating environment to perform desired 
tasks. What’s beyond “smart dust ” 7 and how can it become a material 
resource for informing intelligent systems? 

o A gap exists in the ability to realize material properties and functionalities 
that are tunable and assignable at the point of fabrication (stem cell analog). 
This includes the ability to reassign properties and functionalities (e.g., low 
power recycling to minimize need for new materials) 

o What if the problem of navigating obstacles could be reframed entirely; 
what if the “obstacles” in natural and human-made environments could be 
resources to enable the robot or intelligent system to perform its tasks faster 
and for longer periods of time? Could the obstacles then be seen as assets 
to supply the intelligent system with energy or materials, or enable the 
ability to conserve energy by elastically “bouncing off’ or otherwise using 
the obstacle in innovative ways to propel and navigate the intelligent 
platform around, through or over the obstacle? 

. Knowledge Gaps & Opportunities in Computation and Modeling for 

Intelligent Systems 

o Advancing and implementing the use of computational and analytical tools 
is critical in the integrated design of materials and energy. A knowledge gap 
exists in enabling computation to be brought deeper and more pervasively 
into the materials to enable entirely new, exceptionally rapid, precise, and 
complex response and behaviors. Making intelligent materials and systems 
truly autonomous will require computation 8 . Specifically, the capability to 
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store more than two states and perform simple logic operations 8 . This is 
important, as only co-locating computation with sensing and actuation can 
provide the scalability (with respect to the number of sensors and actuators 
that are integrated in the material) and robustness that make the resulting 
composite a material : unlike a system or device, a material should maintain 
its function when cut into any imaginable shape or when partly damaged. 
Approaches for how to implement such localized computation fall into two 
categories. One approach aims at mimicking digital logic using simple 
chemical or mechanical processes that activate and inhibit each other. These 
basic switches can then be combined into logic gates. A complementary 
approach is to design the composition and geometry of materials such that 
they subject signals passing through them to an arbitrary transfer function. 
Both approaches have been implemented at multiple scales, ranging from 
nano-scale DNA computing, to micro-scale synthetic biology, and macro¬ 
scale lab-on-a-chip like systems. Both approaches are principally Turing 
universal, i.e. able to perform computations of arbitrary complexity. In 
practice, this Turing-universality will only be of theoretical interest due to 
the increasing manufacturing challenges and space requirements of 
achieving more than a few simple logic gates or first-order dynamical 
systems. Hence, a knowledge gap exists in this area. 

o A suite of gaps limiting the availability of desirable modeling capabilities 
were identified in all three breakout sessions: 

■ Concurrent and holistic design of structures and materials 

■ Design tools for multiphysics and multi-length scale 

■ Multiscale and multiphysical optimization and design 

o Coupling active learning with informing material selection and design: The 
idea here was to enable adaptive and robust design of robot actuation by 
simultaneously designing the control system with the structure. Basically, 
can active learning and control (through neural networks etc.) be used 
during the design process so that the robot’s control system has some “idea” 
of its own design and can then adapt during its function? This may enable 
it to function more robustly if it breaks. During the design process various 
scenarios in which parts of the robot break could be simulated, and the 
active leaming/control would have to adapt. The placement of sensors need 
for adaptive feedback may also be added to the optimization process. 

o Multiscale and multiphysical optimization and design: Can we perform 
automated design and optimization of materials and structure of all 
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components of a robot, i.e. energy storage/generation devices, actuators, 
load bearing structures, and multifunctional parts 10 . The multiscale aspect 
would enable design of both materials and structures simultaneously, or at 
least small scale structures that may be helpful for multifunctional designs 
like structural capacitors. The multiphysics aspect would enable 
comprehensive design of various components, especially the design of 
multifunctional parts. Ideally, a comprehensive optimization algorithm 
would be able to design at multiple scales, and be able to use multiphysics 
to decide when it is appropriate to merge functionality, without intervention 
of a human designer deciding which components should be multifunctional 
a priori. The end goal of this concept would be a fully automated design 
optimization algorithm that could design the entire robot with minimal 
human intervention. 

o In order to achieve actuation with robust performance across a wide range 
of operating conditions and modes, it is necessary to develop sophisticated 
design tools to model the multi-physics of the actuator, as well as to produce 
complex models for control. This is especially important for multi¬ 
functional systems which will likely have both local and global control 
spans. 

o Actuator design through multimaterials topology design optimization: 
Topology Optimization (TO) could enable various novel compliant 
mechanical devices like force inverters, mechanical logic gates, etc. 10 
Currently TO designs fully compliant mechanisms, meaning that they are 
made up of one contiguous structure. Can we design TO algorithms to 
combine discrete mechanical components along with compliant 
mechanisms for better performance? 

o Current mathematical models are insufficient for capturing the more 
complex behaviors made possible by confluence of materials, computation, 
power, sensors, etc. 

o How do we enable the integration of material and energy subsystems with 
inherent (and possibly highly beneficial) nonlinearities to get reliable and 
predictable responses and behaviors? 

o The United States has maintained a decisive advantage in software 
programming, including interactive software and devices used in the 
electronic games industry. The breathtaking complexities and creativity 
achieved in near real-time responses between human inputs and virtual 
responses and graphics and interaction between human and computers 
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serves as a fertile model for more pervasive innovation in robotics. How, 
for example, could the creativity in computation, modeling, decision 
making, and control be brought deeper and more holistically into every 
aspect of materials, energy, sensing, and other elements that composes a 
physical intelligent system? What can be learned from academic and 
industrial sectors in their success at constantly stimulating innovation in 
software (including programming for games), and how can this be used to 
stimulate similar innovation and complexity in robotic devices and their 
behaviors? What breakthrough gains in intelligent system performance and 
capabilities could be made at the intersections of game and other software 
advances with material, energy, propulsion, computation, etc.? 

o Multi-scale modelling is extremely complex when bridging just two length 
scales, so it is necessary to embrace some degree of modularity. The 
fundamental balance of efficiency versus flexibility and complexity as a 
function of application is difficult for a multi-function device. It is necessary 
to clearly articulate multi-function objective functions so that design trade¬ 
offs can be made. Additional considerations in the objective function could 
be graceful degradation for some failure modes, as well as highly efficient 
idling behaviors. 

o Models for designing materials to enable behaviors for low Reynolds 
numbers. What bioinspiration can be modeled from insects and 
cmstaceans? 

o Modeling insight needed for materials that combine components with 
multiple scaling relationships so as to maintain overall system performance 

o “Cyber Foam”: local computation and sensing for highly and “exquisitely” 
engineered materials or cellular-like materials and structures to give far 
more significant ranges of low density mechanical response and properties 
than with just engineering materials alone. -Gaps exist also in consideration 
of the responses (e.g., extremely variability for cellular structures over very 
large range of sizes) 

o Template Models for Scaling - How do you break classic trade-offs in 
actuation and motion using materials and mechanisms? E.g. through chaos 
and instability, through scaling and state change. Develop template models 
for scaling to understand how design paradigms shift as a function of size, 
so appropriate actuators and mechanisms can be selected based on function. 

. “Robometrology” 
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It was noted that the ability to measure performance of a given system - including 
intelligent systems and robotics - is a powerful tool to improve their performance. 
This includes basic (6.1) research phenomena. For example, how fast is the material 
response and recovery from a mechanical load, electrical impulse, or chemical 
reaction? Or how much energy and time does it take for an intelligent system to 
perform a task - and how accurately and how reliably is that task repeatedly 
executed? How do we measure efficiencies in intelligent systems using multiple 
(and maybe simultaneous) energy sources? And how can this data be used to assess 
new materials and energy concepts and tradeoffs for robotics both analytically and 
experimentally? 

There are many possible modes for an intelligent system to traverse an irregular 
and challenging span of terrain, including rolling, jumping, snaking, levitating, etc. 
Which of these is most efficient for the objective? And how is “efficient” defined? 
What are the measurable elements and what relationship is there between the 
intelligent system and the environment it interfaces with? 

o Conventional measurements techniques and approaches might be adequate 
for elements of an intelligent system, but may be insufficient in capturing 
and quantifying more complex behaviors and response associated with 
intelligent systems. New methods for quantifying the “goodness” of robotic 
locomotion, propulsion, actuation, and manipulation in terms of response 
times and mass and energy efficiencies could be a powerful metric for 
assessing new concepts and tradeoffs (including new materials) 

o The use of mixed modes of power generation (rather than a single energy 
source) might warrant rethinking the means by which the cumulative 
response and efficiencies of energy usage is made by an intelligent system 

o How could these new robotic metrology methods be implemented into 
computation space for rapid assessment of “virtual” variants - to achieve 
more optimal solutions subject to imposed physical bounds and constraints? 

• Culture Shifts to Promote Interdisciplinary Advancement of Intelligent 

System Science and Technology 

In addition to identifying the knowledge gaps across the themes and thrusts of 
materials, synthesis, and energy for robotics and intelligent systems, the ASPSM 
participants felt strongly that there needs to be new modes for improving 
transdisciplinary research and awareness to enable these advances. This includes 
working across internal and external academic, industrial, and government 
organizations and sectors. While the emphasis of the ASPSM meeting was on 
identifying fundamental knowledge gaps, awareness of trends occurring externally 
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- and the new opportunities and knowledge gaps they create - are worthy of 
consideration even at the basic science level. For example, what will happen 
anyway in terms of technology evolution in general? What are the key advances 
expected in related industries that will likely be available to support the intelligent 
system area? As an example, while Moore’s law is no more, we can still expect 
significant increases in processor performance with reduced power consumption, 
as well as the emergence of photonic integrated circuits that can support novel 
sensing and control systems. This might create unforeseen opportunities in many 
of the areas identified throughout the ASPSM discussions to address “what we can’t 
do now and what we will need to do in the future” in intelligent systems. 

Key Conclusions & Recommendations 

The overarching conclusion was that, indeed, the potential performance benefits of 
intelligent systems and robotic devices are limited by a lack of comprehensive 
innovation and invention that more fundamentally addresses the scale-appropriate 
energy, power, and materials needs of these systems to deliver desired bulk 
responses, behaviors, and capabilities. There were many interesting and relevant 
research areas and thrusts identified with varying degrees of fidelity, and an attempt 
has been made to digest these into five Key Recommendations: 

1. Material and Energy Intelligence in Robotic and Autonomous Systems 

- The Army needs to invest in the science of scale-appropriate energy and 
power strategies unique to intelligent systems. The ASPSM participants 
consistently identified physical constraints associated with current energy 
and power approaches as a critical limitation on intelligent system 
performance and potential. A holistic, comprehensive, and more deeply 
integrated approach to energy is recommended, and would include new 
research for enabling intelligent management, generation, harvesting, 
storage, conversion, and distributed use of energy and power. This includes 
introducing unprecedented levels of adaptivity so the intelligent system can 
develop new symbiotic and other types of energy acquisition relationships 
(e.g., foraging of natural and manmade materials, ambient energy 
harvesting) within its operating environment. It would also include the need 
for new concepts for materials and processes that could contain the reactive 
materials necessary for self-sustainment (e.g., flexible materials to contain 
micro-combustion, catalyst materials for synthetic metabolism to generate 
and consume energy-dense stored chemicals, and modes of planned 
material and structural degradation to enable “self-cannibalism” for 
extending operation under critical conditions). Research investment in these 
new concepts provide the underpinning science foundation for potential 
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agent-agent teaming of different scale intelligent systems to improve 
mission endurance, durability and environmental adaptability. 

2. Disrupt the Notion of an Intelligent System as a “Fixed Platform” - The 

confluence of materials, synthesis, energy, sensing, and computation could 
provide breakthrough approaches to address current limitations to 
intelligent system performance, maneuver, adaptivity, endurance, and 
resilience. Materials and subscale functional elements could be designed to 
promote rapid assembly and synthesis of materials across multi-length 
scales, including materials that can be rapidly disassembled and 
decomposed so they can be repurposed based on the global and evolving 
needs and task demands of the intelligent system. In addition, new research 
is advocated to provide the intelligent system the low power ability to create 
and deploy tools to better perform its assigned tasks and enable its own self- 
sustainment for significant gains in operational endurance. Therefore, Army 
needs to invest in materials research for reconfigurable electronics and 
materials, distributed actuation with localized power, and computationally- 
driven point of need tool synthesis informed by task requirements and 
environmental constraints. 

3. Aggressively pursue research into Multi-Field Metamaterials - In 

recent years there have been burgeoning metamaterial demonstrations, with 
significant successes in complex custom electromagnetic materials for 
optics and antennas, and early demonstrations of passive mechanical 
metamaterials. Advances in microfabrication, synthesis and additive 
manufacturing have matured to where we can start to realize complex multi¬ 
material structures. This, combined with continued progress in simulation 
and reduced cost computation, means it is an opportune time to pursue 
research in multi-field metamaterials with the goals of achieving novel 
materials that move us far beyond the limitations of traditional materials. 
For example, realizing scalable, tunable, highly-efficient actuators 
supporting complex articulation, with potential for integrated sense and 
control, as well as materials for novel platform frames, with unique 
mechanical, optical and electromagnetic properties. The successful 
realization of these broad potential materials will revolutionize autonomous 
platform design, performance and endurance, for example by enabling 
novel high density energy sources to directly drive actuation. In addition, 
these materials have the potential to advance broader Army system 
performance including in the Cyber-Electromagnetic Activities (CEMA), 
communications and platform arenas. 
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4. Cyber Synthesis for Making Good on the Promises of 
Multifunctionality, Bioinspiration, and Point of Use Adaptivity - 

Currently, synthesis and fabrication technologies fail to enable much richer 
and resilient multiscale material functionality and contiguous, defect-free 
morphologies and subscale structures. Such processes also fail at achieving 
more efficient materials interfacing for discrete device/component 
integration. Therefore, the Army needs to invest in exquisite manufacturing 
of small building blocks and infrastructure with multifunctionality and point 
of use adaptivity unique to intelligent systems. For example, fundamental 
advances in synthesis could fully enable the benefits of a “circulatory 
system” to support distribution of alternative high density energy and 
complex processing of energy at the location of consumption. The goal of 
this investment is to enable energy, power, actuation, and computation 
resource elements synthesis in a more distributed and resilient manner as 
needed in the global intelligent system. The benefits of such an investment 
would provide the scientific basis for informing the development of 
synthesis and fabrication methodologies to enable far more complex and 
energy efficient robotic and intelligent system capabilities, responses, and 
behaviors. 

5. Pioneering Opportunities for Computation and Modeling in Intelligent 

Systems - New modes of embedded computation and sensing will be 
equally critical to enriching the potential of material and energy 
performance in intelligent systems. Recent and disruptive advances in 
theoretical and computational material and energy frameworks, including 
“materials by design,” and materials that can provide logic functions and 
sensing can enable “state machines” and disruptively shrink the response 
time and enable efficient new behaviors for maneuver and actuation by 
distributing control within the materials themselves. Similarly, advancing 
the development of artificial intelligence as a tool to assess constraints and 
stretch the solution space for intelligent system design of materials, 
structures, processes, and energy and power cycles could enable leap ahead 
performance gains in mass, volume, and energy efficiencies. 

Summary 

Intelligent systems and robotics are critical technologies both from a strategic U.S. 
military perspective, as well as ensuring a healthy, globally competitive U.S. 
workforce and industrial manufacturing base. In sifting through the extensive 
output from the two days’ worth of brainstorming activities, it was apparent how 
diverse the ideas and insights were, thanks to the equally diverse set of pioneers 
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invited from around the country who helped generate these outcomes. There was 
pervasive evidence and confirmation that there are serious knowledge gaps and 
limits on (if not outright neglect of) concepts in energy, materials, and synthesis in 
the context of robotics and intelligent systems. This includes illuminating 
knowledge gaps for conducting enabling research at the intersections of disparate 
research disciplines. The spirit of the ASPSM meeting can best be summarized in 
one notable outcome from one of the breakout groups. This breakout group first 
worked together to define the current state-of-the-art in robotics and intelligent 
systems, followed by concepts and recommendations for unconventional 
innovation beyond state of the art. This particular group used the playful 
colloquialism “Couch Warrior” to describe the remote operation and control of 
some robotic platforms by Soldiers. Their recommendation for improving the 
performance of Couch Warriors? 

Build a better couch. As humorous and glib as this suggestion is, it is a good 
metaphor for how the “Materials for Sustainable and Mission Flexible Intelligent 
Systems” ASPSM meeting put all ideas, relationships, and aspects in play so that 
more comprehensive approaches to materials, energy, synthesis, and computation 
could broaden the mass, volume, and energy efficient solution space for robotics in 
the future. 
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Distributed Information Processing and Data Analytics 

December 15-16, 2016 
Adelphi Laboratory Center MD 

Organizer: Dr. Tien Pham (ARL/SEDD) and Dr. Alex Kott (ARL/CISD) 

Vision: The Army seeks to understand and exploit the proliferation of “big data” 
gathered by military assets, non-military sensors everywhere, the Internet of 
Things, and social media sites. The sheer volume of this information must be 
distilled from overwhelming levels anticipated to useful and actionable levels using 
a suite of trustworthy, secure, interconnected systems and techniques using 
machine learning and artificial intelligence. 

Objective and Scope: 

The main goal of this ASPSM workshop was to understand near-term and far-term 
implications of “big data”, including but not limited to intelligence analysis 
requirements from key stakeholders and experts, and discuss how to address these 
requirements via R&D in big data processing and analytics. Thus, the ASPSM topic 
aimed at novel capabilities and approaches to address the big data challenges faced 
by the warfighters and decision makers at multiple echelons. 

Focus Areas: 

Topics included, but were not limited to, the following: deep learning and deep data 
analytics, integrated multi-modal information fusion and analytics, social sensing 
and social computing, smart sensing and processing and edge computing platforms, 
devices and services, context-aware information management and resource 
allocation, adaptive distributed data services and scalable and secure information 
infrastructures, and human information interaction (visual analytics, HMI). 

Background: 

The proliferation of “big data” gathered by military assets, non-military sensors 
everywhere (e.g., security cameras, mobile phones), the Internet of Things, and 
social media sites (e.g., blogs, Twitter, Facebook) poses new technical and 
operational challenges, threats and opportunities for the Army. The volume, 
diversity, density, velocity, mobility and degree of distribution of the data will be 
potentially multiple orders of magnitude greater than in today’s world. Decision 
makers at every echelon need to be able to discover, collect, process and make sense 
of the available relevant data for situation understanding and decision-making. In 
order to support the future mission needs of an agile expeditionary force operating 
in complex dynamic environments, the Army must modernize its intelligence 

Approved for public release; distribution is unlimited. 

100 



gathering, processing and analysis capabilities. Entirely new, currently unavailable 
and un-imagined techniques and tactics may become available to both friendly and 
adversary forces. 

Gaps and Recommendations: 

Exploiting Recent Advancement in Artificial Intelligence and Machine Learning 
(AI and ML) 

With the dramatic, continuing increase in digital data generated from sensors 
(physical, electronic, and social) and the continuing growth in computing power 
(both realized and on the near horizon) the Army is at a scientific inflection point 
for the development of game-changing capabilities. Of particular interest is the 
development of portable artificial intelligence capabilities (AI in My Pocket). 
These technologies will produce a new culture by bringing small-scale systems to 
the front line for on-the-fly machine learning and then provide customized, rapid 
training to soldiers on what is learned. Such systems could significantly reduce the 
time it takes units to come up to speed as they learn the lay of new lands and 
cultures. The development of such a capability will require many scientific 
breakthroughs and discoveries. On the technological side, these pocket sized 
systems will need to be developed with sufficient computing, memory, and storage 
power to function on their own as well as part of the network to leverage the power 
of nearby devices in forming a processing cluster. On the scientific side, research 
must be performed to exploit the extent to which AI can replicate, compliment, and 
improve the human brain for information processing, sense making, and decision¬ 
making. Ultimately, such advancements can provide soldiers with near 
instantaneous knowledge to make near-immediate sense of complex environments 
and situations. 

In addition to the science and technology developments of artificial intelligence and 
machine learning for the physical world, such advancements are equally, if not 
more pressingly, necessary in the cyber world. In this domain, AI and ML 
advancements are necessary to penetrate and learn an enemy’s AI systems, to 
exploit their AI, to improve our AI, and to protect our AI. Important scientific 
problems yet to be solved in this space include the ability for one AI system to learn 
from another; the ability for an AI to validate that another AI is behaving as 
expected and not working outside its parameters; and the ability for an AI system 
to move without detection for stealth penetration and learning. 

Vulnerability and Adversarial Effects in Data, Analytics Tools, and Training 

Novel approaches are particularly required in ill-structured problems characterized 
by complexity, emergent and inconsistent behaviors, ambiguous cause and effect, 

Approved for public release; distribution is unlimited. 

101 



and highly dynamic environments and behaviors. However, these are precisely the 
areas where modem AI and Machine Learning approaches are least developed, and 
where opportunities for adversarial deception of the machine processing are most 
plentiful. This is further complicated by the fact that no machine reasoning occurs 
without interaction with humans (users, developers, maintainers, etc.). These 
phenomena in human-machine systems are also notoriously difficult to understand, 
predict and mitigate. Approaches are needed to make machine learning and 
reasoning more resilient and resistant to deception, false facts, biases and 
influences. It might be possible that exploiting a high degree of connectivity within 
the future battlefield, heavily saturated with sensors and intelligent 
machines/munitions, will provide opportunities for cross-validation of facts and 
inferences. Approaches are also needed to ensure that conclusions, indicators and 
warnings provided by intelligent data processors are consistent with human 
decision-making needs and are not counter-productive with respect to human 
cognitive processes. Novel methods for validation and verification will need to 
emerge to ensure adequate quality of operations in such complex human-machine 
systems in the environments with such ill-structured, dynamic and deceptive 
problems. 

Exploitation of OSINT and Non-traditional sources 

The amount of easily accessible “free” information has grown exponentially with 
the proliferation of social media sites and apps. The use of open-source intelligence 
(OSINT) is not just a good idea, but a necessity for national security. Traditional 
OSINT sources include but are not limited to social media such as podcasts, 
YouTube, Facebook, Twitter; news sources; scientific articles; government 
documents such as patents and disclosures; mass transit maps and schedules; and 
the dark web. Non-traditional OSINT sources can include data about city 
infrastructure such as power usage, emergency responders, and hospital activity; 
cameras; traffic patterns; mobile phone usage; and from the human terrain 
information about culture, norms, teams, organizational structures, and power 
structures. 

The primary challenges and opportunities in this space include (1) developing a 
means to usefully store the volume of data, discerning what to keep and what to 
delete, and making parts, if not all, of it portable and field ready; (2) accurately 
identifying the signal amongst the noise in the massive amount of information; and 
(3) data fusion to intelligently bring together various sources and types of data such 
as text, image, audio, video, sensor, AI, time, location, and person as well as 
combining OSINT with HUMINT and other intelligence information. Additional 
challenges and opportunities include not only tracking but predicting information 
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propagation; machine learning to make sense of OSINT; identification of unknown 
unknowns and novel activities before they become damaging events; and the 
mapping of human networks with name/identity (dis)aggregation for accurate 
modeling. 

To increase confidence in an OSINT system, analysts must develop a sense of trust 
in how data are collected, entered, stored, accessed, and analyzed in the system. To 
assist in trust-building, data must be vetted with regard to its accuracy. 
Additionally, the system must be able to maintain provenance information so that 
analysts can always trace information to its source, determine who entered it into 
the system, and see who accessed it along the way. Moreover, trust in the use of 
OSIST not automatic, but will be developed through consistent demonstration and 
improvement of the exploitation of these data. Such exploitation will include the 
use of social network analysis for social influence prediction and prevention; cross- 
cueing to focus attention on where to look; successful data fusion and validation; 
correctly identifying the signals amongst the noise; dynamic constraint-based query 
abilities; on-the-fly machine learning capabilities to develop common ground and 
ad hoc questioning; and lastly, moving from analysis of past events to prediction. 

Recognizing Emergent Phenomenon and Pattern of Life sooner than Human 
Analysts 

The implications of the operational environment in 2030 from recent TRADOC 
studies are contested domains and degraded operations, lethal battlefield, complex 
terrain and challenged deterrence. In the near future, dense urban areas or 
megacities will be an operational environment. Can machine learning approaches, 
algorithms and systems help make sense of complex situations, recognize emergent 
phenomenon faster than any single or group of analysts/soldiers could, and 
importantly, overcome confirmation bias? 

The analysts and soldiers are predominately dealing with ill-structured problems 
which are highly unpredictable and have: (1) emergent behaviors and (2) complex 
relationships with freedom of action and interaction, ambiguous cause and effect, 
inconsistent behavior, and uncertainty. Can machine learning potentially detect and 
identify key features to make sense of complex relationships? Current Intelligence 
Preparation of the Battlefield (IPB)* will not be effective against emerging 
phenomena in which complex entities and patterns arise through interactions 
among simple entities that themselves do not exhibit such properties. Can 
computer-driven intelligence agents be used to analyze and predict emergent 


* IPB is the systematic process of analyzing the mission variables of enemy, terrain, weather, and 
civil considerations in an area of interest to determine their effect on operations). 
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phenomena? Lastly, a risk with data mining and/or pattern analysis conducted by 
human intelligence analysts is that the human becomes motivated to “discover” 
patterns that are either meaningless or simply not there. Can machine learning 
systems add significantly to the analytic rigor since it looks at all the data and not 
just the data it believes to be relevant or able to explore a greater number of 
hypotheses, many of which might escaped human attention? 

Infrastructure, Methods, Standards, Tools to Make Advanced Analytics Rapidly, 
Reflectively and Deployable 

Research to make relevant advancements in big data processing and analytics and 
to rapidly transition analytics technology to the end users (analysts or warfighters), 
flexible infrastructure environments with sufficient diversity is needed to 
accommodate a variety of applications, domains and users. However, there are a 
number of challenges associated with developing a flexible data analytics 
infrastructure: (1) standardization of data and ground truths, analytic algorithms, 
software tools, performance metrics, and validations; (2) physical infrastructure 
that scales in terms of processing/computer power and classification levels; (3) 
operating infrastructure in terms of format/primitives, training and testing 
environment (batch or streaming); (4) validation and verification of 
CONOPS/doctrines at different classification levels on adaptive learning and 
evolving complex systems. At the minimum, the infrastructure should create a 
highly collaborative environment with non-propriety open architecture system, 
relevant data sets with ground truths, provide access to operational and domain 
experts and scalable computational resources, and enable experimentation for 
research validation and technology transition. A side benefit to having access to 
such an infrastructure facility is the education and training of future data scientists. 

Conclusion: 

The development of distributed information processing and data analytics 
technologies and techniques has the potential for significant payoff in future Army 
systems. Harnessing advances in artificial intelligence and machine learning will 
provide tomorrow’s soldier with the tools to extract critical information from the 
otherwise overwhelming deluge of information on the battlefield. Focusing this 
research on the key Army issues will leverage commercial advances and accelerate 
system development, with the strong potential for leap ahead capability 
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